Window display device with a three-dimensional orientation of windows

ABSTRACT

A storage unit  5102  stores image data for a window, and a three-dimensional position calculating unit  5104  calculates a position for the window in a virtual three-dimensional space so that it appears to be at a position inclined in the depth direction. A perspective projection unit  5106  transforms image data so that a window positioned at the calculated coordinates is projected onto a screen in perspective, taking a point in front of the screen as a viewpoint. A frame memory unit  5107  holds image data for the window on which perspective projection has been performed, and a display unit  5108  displays the window on the screen.

INDUSTRIAL FIELD OF USE

The present invention relates to a window display device for displayingwindows on the screen of an information processing apparatus, and inparticular to a device for displaying windows in a virtualthree-dimensional (3D) space.

BACKGROUND ART

Multiple window display devices are used to increase the performance ofthe man-machine interface in computer information terminals and thelike.

One example of a prior art technique is disclosed in Japanese Laid-OpenPatent No 6-222899. Here, a multiple window display device improvesdisplay efficiency by displaying as many windows as possible in thelimited screen space available.

FIG. 69 shows a display screen for this prior art example.

In the prior art, windows are placed in a virtual 3D space, with windowspositioned deep within the 3D space reduced in size. As a result, morewindows can be displayed, making the screen appear larger than itactually is.

However, in this prior art example, a window placed deep within the 3Dspace is displayed reduced in size, making it impossible to distinguishany of the display content of the window.

DISCLOSURE OF THE INVENTION

The present invention, a device for displaying windows placed in avirtual 3D space, is designed to overcome the stated problem. The objectof the invention is to provide a window display device that increasesthe efficiency with which windows can be displayed, making the screenappear larger than it actually is, and moreover, preventing a situationin which the entire display content of a window is unreadable.

The present invention aiming to achieve this object is a window displaydevice displaying a plurality of windows on a screen. The window displaydevice includes the following. A display data holding unit holds displaydata representing display content of the plurality of windows. Aposition determining unit determines a position for each window in avirtual three-dimensional (3D) space, the positions being such that thewindows are not parallel to the screen. A perspective projection unittransforms the display data to project the windows onto the screen inperspective based on the determined positions. A display unit thendisplays the transformed display data on the screen.

This enables at least one window to be displayed inclined in the depthdirection of a virtual 3D space. Even if the window is positioned deepwithin the 3D space, the problem apparent in the prior art, wherereducing the size of the whole window by the same amount meant that allof the window display content became unreadable, can be avoided. Thescreen display area can also be used more effectively, and the screenmade to appear larger than it actually is. Furthermore, information inthe inclined window that is near to the front of the 3D space remainseasily recognizable, and less display area needs to be used for the partof the window deep in the 3D space. Thus a window display apparatuscapable of using the screen display area effectively, while ensuringthat at least one part of the window display content is readable can berealized.

Here, the window display device further includes an input unit acquiringindication from a user, enabling the position determining unit todetermine positions according to indications acquired from the inputunit. This enables the user to determine the size of the entire windowand the degree at which it slants in the depth direction.

The position determining means also determines 3D coordinates for eachvertex of a window as the position for the window. If the window is arectangular plane, this enables the position of the window in the 3Dspace to be determined simply by the coordinates for its four vertices.

The window display device further includes a window analyzing unitdetecting, as a key part for each window, a part of the display contentfulfilling certain conditions. Detection is performed based on displaydata held by the display data holding unit. This enables the positiondetermining unit to determine the position of windows so that the keyparts detected by the window analyzing unit are displayed near the frontof the virtual 3D space. The window can thus be displayed inclined inthe depth direction, so that the key part of the display content is nearthe front of the 3D space. Therefore, even if a plurality of windows arebeing displayed, the user can grasp the type and display content of eachwindow at a glance.

The window analyzing unit also determines whether a title is included inthe display data for each window and, if a title is included, detects apart of a window in which the title is displayed as the key part. Thisenables the user to easily recognize window types.

The window analyzing unit also determines whether a text and anindication for displaying the text horizontally are included in thedisplay data for each window, and if the text and the indication areincluded, detects a left side of a window when viewed from the front asthe key part. This enables the user to easily recognize the type andgeneral content of a horizontally-written document displayed in awindow.

The window analyzing unit also determines whether a text, and anindication for displaying the text vertically are included in thedisplay data for each window, and if the text and the indication areincluded, detects a right side of a window when viewed from the front asthe key part. This enables the user to easily recognize the type andgeneral content of a vertically-written document displayed in a window.

The position determining unit includes an automatic alignment unit fordetermining the position of a plurality of windows so that the windowsare placed in a predetermined arrangement, where at least one window isnot parallel with the screen. This enables a plurality of windows thatwere displayed scattered around the screen to be displayed in alignment,using the screen display area more effectively and allowing all of thewindows currently displayed to be viewed at a glance.

The position determining means further includes a unit determining aposition according to an indication acquired from the input unit, sothat a window displayed in the alignment position is newly displayedfacing the front; and a unit determining a position according to anindication acquired from the input unit, so that a window displayedfacing the front is returned to the alignment position. This enables aoperation, in which a window faces the front when it is being operatedon, and returns to the alignment position once it is no longer beingoperated on, to be easily performed.

The window display device further includes a window information imageposition changing unit editing display data stored in the display dataholding unit so that a window information image is displayed near thefront of the virtual 3D space. The window information image is formedfrom a title bar and a menu bar, and editing is performed based on aposition determined by the position determining unit. This enables thetitle bar and menu bar for a window to be displayed near the front ofthe 3D space, even if the window is inclined in the depth direction. Asa result, recognizing window types and performing window operationsbecomes easier.

The window display device further includes a window information imageorientation changing unit controlling the position determining unit sothat, in a window where a window information image is displayed near thefront of the virtual 3D space, the window information image faces thefront, while the remaining part of the window :is not parallel with thescreen. This enables the title bar and menu bar to be displayed facingthe front near the front of the virtual 3D space, even if the window isinclined in the depth direction. As a result, recognizing window typesand performing window operations becomes easier.

The window display device further includes the following. An unreadablearea determining unit determines whether an unreadable area displayingcharacters smaller than a specified size has been generated from displaydata transformed by the perspective projection unit. A scroll buttonposition determining unit determines a position for a scroll button whenan unreadable area has been determined to exist, the position determinedas if the unreadable area was not displayed in the window. A scroll barattachment unit edits the display data so that a scroll bar image,including a scroll button arranged at the determined position, isattached to the window. This enables characters and the like in anunreadable area of a window displayed inclined in the depth direction tobe scrolled up into the readable display area.

The window display device further includes the following. A layerseparating unit separates display data held in the display data holdingunit, the display data corresponding to a window with a layeredconstruction having a base layer and at least one upper layer, and thedisplay data for each upper layer, but not the base layer, beingseparated. A window layer generating unit generates display data for awindow layer corresponding to each separated layer. The positiondetermining unit determines a position for each window layer parallelwith and a specified distance from the base layer. This enables windowlayers to be displayed floating over the base layer so that objectsdisplayed on each layer can be grasped at a glance, and editing of anobject across windows can be easily performed.

The window display device further includes the following. A keywordsearching unit searches display data held in the display data holdingunit for parts that match a keyword acquired from the input unit. Awindow layer generating unit generates display data for a new windowlayer in which the keyword is arranged at positions corresponding to thematching parts, and stores the generated display data in the displaydata holding unit. Here, the position determining unit determines aposition for the window layer parallel with and a specified distancefrom a display object window. This enables a window layer showing thematching parts to be displayed floating over the original window. Theuser's attention is focused on the window layer only, allowing theposition of the keyword to be confirmed.

The window display device further includes the following. Aninter-related window specifying unit specifies a group of inter-relatedwindows from the plurality of displayed windows. A link informationstorage unit generates and stores link information used to line up thegroup of inter-related windows in a concertina arrangement with framesof neighboring windows touching. Here, the position determining unitdetermines positions of the group of inter-related windows based on thelink information. This enables a plurality of document pages displayedin different windows to be displayed in a linked sequence, so that, forexample, windows belonging to the same document can be easilydistinguished.

The window display device further includes the following. A linkinformation changing unit changes the link information according to anindication acquired from the input means, so that the group ofinter-related windows is lined up with part of the group excluded. Here,the position determining unit determines positions for the group ofinter-related windows, apart from the excluded part of the group, basedon the changed link information. This enables the windows that the userwants to focus on from the linked sequence of windows to be displayed.

The window display device further includes an inter-window editing unitediting display data held in the display data holding unit according toan indication acquired from the input unit, so that editing of displaycontent can be performed between two windows in the group ofinter-related windows. This enables the windows that the user wants tofocus on from the linked sequence to be displayed adjacent to eachother, and editing of an object performed between the two windows, sothat the efficiency of operations performed spanning a plurality ofwindows is improved.

The display data holding unit includes a character data storage unitstoring character data formed of character information displayed in thewindows, and a window data storage unit storing window data formed ofinformation concerning non-character information displayed in thewindows. Here, the position placement determining unit determines aposition for each window and the corresponding characters according toan indication acquired from the input unit, and the perspectiveprojection unit includes the following. A perspective transform unitgenerates two-dimensional data showing the vertices and window imagedata by performing a perspective projection transform on the windowdata, the perspective projection transform being performed based onwindow positions determined by the position determining unit. Acharacter data transform unit generates character image data bytransforming the character data, based on character positions determinedby the position determining unit and two-dimensional data showingvertices obtained by the transform performed by the perspectivetransform unit. A combining unit combines image data generated by theperspective transform unit and image data generated by the characterdata transform unit. Here, the display unit displaying image datacombined by the combining unit.

The character data storage unit includes a character information storageunit storing information relating to characters displayed in the window,the information including at least character position informationshowing positions of the characters in virtual space, characteridentifying information specifying which characters should be displayed,and character size information, and a character image storage unitstoring character images. Here, the window data storage unit storesgraphics data, comprising window display content excluding characters,and window coordinate information showing a position of the window inthe virtual 3D space, the position determining unit changes the windowdata and the character position information in response to an indicationacquired from the input unit and the perspective transform unit performsa perspective projection transform on window data that has been changedby the position determining unit. The character data transform unitincludes the following. A character position information conversion unitconverts character position information changed by the positiondetermining unit into screen position information. A character imageselecting unit selects a character image from the character imagesstored in the character image storage unit, the character imagecorresponding to the character identifying information. A characterimage transform unit transforms the character image selected by thecharacter image selecting unit in accordance with the result of theposition changing process performed by the position determining unit.Here, the combining unit combines a character image transformed by thecharacter image transform unit with image data generated by theperspective transform unit, so that the character image is placed at aposition in the window shown by the screen position information producedby the character position information conversion unit, the window havingbeen transformed onto the screen by the perspective transform unit.

The character image transform unit includes a shape transform unittransforming the shape of the character image using a similaritytransform related to the shape of the window transformed by theperspective transform unit; and a scaling unit finding a scale based oncharacter position information stored in the character informationstorage unit, and changing the size of the character image according tothe scale. Here, character images are separated from the window as 2Ddata, their shape transformed by a similarity transform with theperspective-transformed window, and their size converted according to ascale calculated from coordinate values. After being transformed into ashape similar to the perspective transform result, the character imagesare combined with the perspective-transformed window. This processingoccurs rather than having character images rotated andperspective-transformed after they have been developed into virtual 3Ddata. When compared to the prior art, in which a window includingcharacter images was rotated and then perspective-projected, the amountof processing is reduced, and performance improved. The amount of dataused for the character images is also decreased, reducing the amount ofmemory required.

The character image transform unit calculates a scale based on characterposition information stored in the character information storage unit,and changes the size of the character image according to the scale. Thisenables only character image size to be changed by the scale found fromthe character position coordinates, further reducing the amount ofprocessing required and creating a sense of depth when characters aredisplayed in a window as a character string.

The character image transform unit includes an aspect ratio conversionunit finding an aspect ratio based on character position informationstored in the character information storage unit and changing the lengthto width ratio of the character image according to the aspect ratio, anda scaling unit finding a scale based on character position informationstored in the character information storage unit, and changing the sizeof the character image according to the scale. This enables characterimages to be transformed using an aspect ratio, rather than simply byusing a scale. As a result, a more natural sense of depth can beachieved when character images are displayed as a character string in awindow than was the case when only the size of character images waschanged according to a scale.

The character image storage unit stores character images on which asimilarity transform has been performed for each of a plurality ofrepresentative positions for a window in the virtual 3D space, thesimilarity transform being related to the shape of the window producedat each of the plurality of positions as a result of a perspectiveprojection transform. The character image selecting unit selects acharacter image from the character image storage unit, the characterimage coinciding with character identifying information stored in thecharacter information storage unit, and with window coordinateinformation stored in the window data storage unit. The character imagetransform unit finds a scale based on character position informationstored in the character information storage unit, and changes the sizeof the character image selected by the character image-selectingunit,.the size of the character image being changed according to thescale. This enables character images having a natural sense of depth andsimilar to character images that have been perspective-transformed, tobe displayed simply by changing the size of pre-prepared characterimages.

The window display device further includes a character image readabilityjudging unit. This unit calculates a size for a transformed characterimage transformed by the character image transform unit from sizeinformation stored in the character information storage unit, and ascale calculated from character position information stored in thecharacter information storage unit. It then judges whether thetransformed character image is readable by comparing the calculated sizewith a threshold value, and informs the character image selecting unitof the judgement result. Here, the character image storage unit furtherstores symbol images, the character image selecting unit selects aspecified symbol image from the character image storage unit to bedisplayed instead of a character image if the character imagereadability judging unit judges that the character image is unreadable,the character image transform unit does not perform transform processingon the symbol image, and the combining unit combines the symbol imagewith image data generated by the perspective transform unit so that thesymbol image is placed at a position in a window shown by screenposition information produced by the character position informationconversion unit, the window having been transformed onto the screen bythe perspective transform unit. This enables a symbol to be displayedinstead of characters considered to be difficult or impossible to readif transformed. As a result, processing for reading and transformingcharacter images is shortened, reducing both processing volume and theamount of time required for processing.

The character information storage unit further stores font typeinformation showing character font types. The character image storageunit stores character images for each font type. The window displaydevice further includes a character image readability judging unit. Thisunit calculates a size for a transformed character image transformed bythe character image transform unit from size information stored in thecharacter information storage unit, and a scale calculated fromcharacter position information stored in the character informationstorage unit. It judges if the font type of the transformed characterimage is readable by comparing the calculated size with a thresholdvalue, and informs the character image selecting unit of the judgementresult. Here, when the font type of a character is judged unreadable bythe character image readability judging unit, the character imageselecting unit selects a character image with a specified font type,ignoring font type information stored in the character informationstorage unit. This enables individual fonts to be ignored in the case ofcharacter images whose display font will not be discernable iftransformed. Instead these characters are displayed in an identicalspecified font. If cache memory is used, the cache hit rate when thefont is read increases, improving processing efficiency.

The window data storage unit further stores coordinate systeminformation showing a virtual space window coordinate system, and thewindow display device further includes a coordinate system convertingunit for changing a scale of a coordinate axis showing screen positions,the scale being changed according to window coordinate informationstored in the window data storage unit. Here, the perspective projectionunit performs a perspective projection transform, converting thecoordinate system to a coordinate system having a coordinate axis withthe changed scale. This enables the scale of the coordinate system to bechanged, so that even characters far from the rotation axis, which wouldbe cramped and difficult to read in a conventional coordinate system,can be displayed so that they are easy to read.

BRIEF EXPLANATION OF THE DRAWINGS

FIG. 1 is a block diagram showing a structure for a multiple windowdisplay device in the first embodiment of the invention;

FIGS. 2A and 2B illustrate the concept of perspective projection,performed by a perspective projection unit 5106,

FIG. 2A shows the positional relationship of the center of theprojection, the projection screen, and the object (a window);

FIG. 2B shows a perspective view produced when the object is solid;

FIGS. 3A to 3C show the actual processing content of the perspectiveprojection performed by the perspective projection unit 5106;

FIG. 3A shows the user's viewpoint, the positional relationship ofdisplay screen and window, and the shape of the perspective-projectedwindow when the window placed in the virtual 3D space is parallel to thedisplay screen;

FIG. 3B shows the user's viewpoint, positional relationship of displayscreen and window, and the shape of the perspective-projected windowwhen the window placed in the virtual 3D space is not parallel to thedisplay screen;

FIG. 3C shows actual calculation processing for the perspectiveprojection;

FIG. 4 is a flowchart showing operation processing for the windowdisplay device in the first embodiment;

FIG. 5 is an example display screen in which windows 5130 and 5131 arearranged parallel to the screen and windows 5132 and 5133 are arrangedin a non-parallel position;

FIG. 6 is a block diagram showing a structure for a multiple windowdisplay device in the second embodiment;

FIG. 7 is a flowchart showing the processing performed when a windowanalyzing unit 5201 specifies a key part of a window;

FIG. 8 is an example display screen occurring when the key part has beenspecified by the window analyzing unit 5201;

FIG. 9 is a block diagram showing a structure for a multiple windowdisplay unit in the third embodiment;

FIG. 10 is a flowchart showing automatic alignment processing performedby an automatic aligning unit 5104;

FIG. 11A shows an example of a first alignment method in which windowsare not stacked;

FIG. 11B shows an example of a second alignment method in which windowsare stacked;

FIG. 12A shows an example of a display screen immediately afterautomatic alignment has been performed;

FIG. 12B shows an example of a display screen immediately after a window5241 displayed in the display screen of FIG. 12A has been selected by aninput unit 5103;

FIG. 12C shows as example of a display screen immediately after a windowadjustment button 5245 on the right edge of the title bar of the window5241 in the display screen of FIG. 12B has been pressed using the inputunit 5103;

FIG. 13 is a block diagram showing a structure for a multiple windowdisplay device in the fourth embodiment;

FIG. 14 is a flowchart showing operation processing for a windowinformation image position changing unit 5301 and a window informationimage orientation changing unit 5302;

FIG. 15 shows an example display screen containing a window 5330, priorto changing the position of the window image information and a window5331, after changing the position of the window image information;

FIG. 16 shows an example display screen containing a window 5332, priorto changing the orientation of the window image information and a window5333, after changing the orientation;

FIG. 17 is a block diagram showing a structure for a multiple windowdisplay device in the fifth embodiment;

FIG. 18 is a flowchart showing operation procedures for a scroll buttonposition determining unit 5401, a scroll bar attaching unit 5402 and anunreadable area determining unit 5403;

FIG. 19A shows the basic concept for a scroll button 5430 and a scrollbar 5431;

FIG. 19B shows an example display screen when the lower half of thewindow display area has been determined to be an unreadable area 5432;

FIG. 20 is a block diagram showing a structure for a multiple windowdisplay device in the sixth embodiment;

FIG. 21 is a flowchart showing operation procedures for a window layerseparating unit 5501 and a window layer generating unit 5502;

FIG. 22A shows an outline of the layer separation processing occurringwhen there are two layers;

FIG. 22B is an example display screen occurring when a layered window isarranged inclining in the depth direction;

FIG. 23 is a block diagram showing a structure for a multiple windowdisplay device in the seventh embodiment;

FIG. 24 is a flowchart showing operation procedures for a keyword searchunit 5601, a search result output unit 5602 and a window layergenerating unit 5603;

FIG. 25 shows an example of a display screen when a keyword searchresult is displayed as a window layer;

FIG. 26 is a block diagram showing a structure for a multiple windowdisplay device in the eighth embodiment;

FIG. 27 is a flowchart showing operation procedures for a window linkingunit 5701, a window positional relationship storage unit 5702, and awindow positional relationship changing unit 5703;

FIG. 28A is an example display screen on which windows page 1 to page 5are displayed based on initial link information;

FIG. 28B shows an example display screen when link information ischanged so that the window of page 3 is folded up;

FIGS. 29A and 29B show examples of a display screen when a means forperforming an operation extending over more than one page of the linkedpage windows on an object is provided;

FIG. 29A is identical to FIG. 28A and shows a display screen whenwindows for pages 1 to 5 are displayed based on initial linkinformation;

FIG. 29B shows a situation in which an object from page 1 is copied topage 4 after pages 2 and 3 have been folded up so that pages 1 and 4 aredisplayed as facing pages;

FIG. 30 is a block diagram showing a ninth embodiment for the windowdisplay device of the present invention;

FIG. 31 shows an example of an object placed in virtual space in theninth embodiment;

FIG. 32 shows an example of window information in the ninth embodiment;

FIG. 33 shows an example of character information in the ninthembodiment;

FIG. 34 shows an example of window rotation and perspective transform inthe ninth embodiment;

FIG. 35 shows an example of window information produced after rotationprocessing has been performed in the ninth embodiment;

FIG. 36 shows an example of character information produced afterrotation processing has been performed in the ninth embodiment;

FIGS. 37A and 37B show examples of character and window informationafter a perspective transform in the ninth embodiment;

FIG. 38 shows an example of a perspective transform for a window priorto rotation processing in the ninth embodiment;

FIG. 39 shows an example of a perspective transform for a window afterrotation processing in the ninth embodiment;

FIG. 40 shows the relation between Z coordinate values and font scale inthe ninth embodiment;

FIG. 41 shows examples of sizes of display fonts in the ninthembodiment;

FIG. 42 is a flowchart showing window display processing in the ninthembodiment;

FIG. 43 is a block diagram of a window display device in the tenthembodiment of the present invention;

FIG. 44 shows an example of a perspective transform view of a windowprior to rotation processing in the tenth embodiment;

FIG. 45 shows an example of a perspective transform view of a windowafter rotation processing in the tenth embodiment;

FIG. 46 is a flowchart of window display processing in the tenthembodiment;

FIG. 47 is a block diagram of a window display device in the eleventhembodiment of the present invention;

FIG. 48 shows an example of a perspective transform view of a windowprior to rotation processing in the eleventh embodiment;

FIG. 49 shows an example of a perspective transform view of a windowafter rotation processing in the eleventh embodiment;

FIGS. 50A to 50C show the relation between window position and aspectratio, and between character position and size, in the eleventhembodiment;

FIG. 51 is a flowchart showing window display processing in the eleventhembodiment;

FIG. 52 is a block diagram of a window display device in the twelfthembodiment of the present invention;

FIGS. 53A to 53F are examples of font characters stored in the fontdatabase in the twelfth embodiment;

FIG. 54 is a flowchart of window display processing in the twelfthembodiment;

FIG. 55 is a block diagram of a window display device in the thirteenthembodiment of the present invention;

FIG. 56 is an example of a perspective transform view of a window beforerotation processing is performed in the thirteenth embodiment;

FIG. 57 is an example of a perspective transform view of a window afterrotation processing is performed in the thirteenth embodiment;

FIG. 58 is a flowchart showing window display processing in thethirteenth embodiment;

FIG. 59 is a block diagram of a window display device in the fourteenthembodiment of the present invention;

FIG. 60 shows an example of a perspective transform view of a windowbefore rotation processing is performed in the fourteenth embodiment;

FIG. 61 shows an example of a perspective transform view of a windowafter rotation processing is performed in the fourteenth embodiment;

FIG. 62 is a flowchart of window display processing in the fourteenthembodiment;

FIG. 63 is a block diagram of a window display device in the fifteenthembodiment of the present invention;

FIGS. 64A to 64D show examples of perspective transform views of windowsoccurring in the fifteenth embodiment;

FIGS. 65A to 65D are examples of window display coordinate systemsbefore and after conversion in the fifteenth embodiment;

FIGS. 66A to 66D are examples of window displays when the presentinvention is applied to a gray-scale display font;

FIGS. 67A and 67B show display screens when the present invention isapplied to windows displayed on the surfaces of a solid object;

FIGS. 68A and 68B show examples of the application of the presentinvention in actual electronic products; and

FIG. 69 is an example of a multiple window display screen in the priorart.

BEST MODE FOR CARRYING OUT THE INVENTION

The following is a description of the embodiments of the invention, withreference to the drawings.

First Embodiment

First, a multiple window display device in the first embodiment isexplained.

The multiple window display device is characterized by the ability todisplay windows inclined in the depth direction of the screen (hereafteralso described as the ability to display windows in perspective) as ifplaced in a virtual 3D space. A window is a rectangular plane, theoutline of which is formed by specifying four vertices.

Structure

FIG. 1 is a block diagram showing a structure for a multiple windowdisplay device in the first embodiment.

The multiple window device includes a program execution unit 5101, astorage unit 5102, an input unit 5103, a 3D position calculating unit5104, a texture mapping unit 5105, a perspective projection unit 5106, aframe memory 5107 and an image display unit 5108.

The program execution unit 5101 is a CPU or similar, which activates andexecutes application programs that interact with users by displayingconventional two-dimensional (2D) windows. The program execution unit5101 stores display data representing content to be displayed in eachwindow in the storage unit 5102. This display data includes code dataspecifying window display content (documents, characters, graphics etc.)and image data expressing the objects as bitmap images. The programexecution unit 5101 exchanges information regarding the windows with theother structural components 5201 to 5108. The image data is of apredetermined fixed size and is not dependent on the size of an entirewindow or on the position of a window within the 3D space.

The storage unit 5201 is memory for storing the display data, that iscode data and image data, corresponding to each window in the variousapplication programs. All the windows stored in the storage unit 5201need not be displayed on the image display unit 5108, but all windowsdisplayed on the image display unit 5108 must be stored in the storageunit 5102.

The input unit 5103 is a mouse or pointer that obtains user input.

The 3D position calculating unit 5104 calculates a position (coordinatesfor the four vertices of the window) in the 3D space for a window storedin the storage unit 5201 and stores the result according to anotification from the program execution unit 5101 or the input unit5103. In fact, when the 3D position calculating unit 5104 receivesnotification of four vertex coordinates from the program execution unit5101 as an initial position, it stores the coordinates withoutperforming any calculation. When the 3D position calculating unit 5104receives notification of a displacement (type and amount ofdisplacement) in 3D space for a window that has already been positioned,however, it calculates new vertex coordinates using an Affine transform.

If the 3D position calculating unit 5104 does not receive a clearindication regarding the positioning of the window from either theprogram execution unit 5101 or the input unit 5103, it generates andstores a default position. This default position may be, for example, aposition in which the left edge of the window is used as a rotation axisand the right edge inclined at only 45° in the depth direction.

When a new position is calculated by the 3D position calculating unit5104, the texture mapping unit 5105 reads image data for the window fromthe storage unit 1502 and reduces or enlarges the data according to theposition. The texture mapping unit 5105 performs texture mapping, thatis processing for combining and pasting the image data for all theobjects making up the window, and transmits the texture data obtainedfrom the processing to the perspective projection unit 5106. The texturedata is determined only according to the position of the window in 3Dspace and not according to the viewpoint from which the window isobserved.

The perspective projection unit 5106 performs perspective projection onthe texture data transmitted from the texture mapping unit 5105 andstores the data obtained in the frame memory 5107. Here, ‘perspectiveprojection’ is an operation for generating a perspective view of anobject (window) placed in the 3D space. A viewpoint in front of thedisplay screen of the image display unit 5108 is taken as the origin forthe projection and the 2D display screen as the projection surface. Inother words, image data is converted from a virtual 3D coordinate systemto a screen coordinate system. In this description, the terms ‘near thefront’, ‘at the back’ and ‘in the depth direction’ are expressed fromthe viewpoint of a user positioned in front of the display surface.

FIGS. 2A and 2B illustrate the concept of the perspective projectionperformed by the perspective projection unit 5106.

FIG. 2A shows the positional relationship between the projection origin,the projection surface and the object (window).

The perspective projection unit 5106 locates the display screen of theimage display unit 5108 between the window positioned in the virtual 3Dspace and the viewpoint of the user. The perspective projection unit5106 causes an image to be shown on the display screen by shining beamsof light at the window. The light beams converge at the viewpoint. Thisgenerates a window on which perspective projection has been performed.

FIG. 2B shows the perspective view when the object is solid.

In this case, a point on a infinite horizon line at which parallel linesmeet (the vanishing point) exists to the rear of the solid object. Sincethe perspective projection unit 5106 performs multipoint rather thanone-point perspective projection, coordinates for 256 vanishing pointsplaced at predetermined positions in the virtual 3D space are stored.

FIGS. 3A to 3C illustrate the processing content of perspectiveprojection performed by the perspective projection unit 5106.

FIG. 3A shows the viewpoint of the user, the positional relation betweenthe display screen and the window, and the shape of a window on whichperspective projection has been performed, when a window located in thevirtual 3D space is parallel to the display screen.

In this case, the perspective-projected window is a rectangle.

FIG. 3B shows the viewpoint of the user, the positional relation betweenthe display screen and the window, and the shape of a window on whichperspective projection has been performed, when a window located in thevirtual 3D space is not parallel to the display screen, in other wordswhen the window is tilted with respect to the screen and displayed inperspective.

In this case, the perspective-projected window is a trapezoid.

FIG. 3C illustrates actual calculation processing for the perspectiveprojection.

The display screen is an xy plane, whose depth direction is shown by az-axis. The drawing shows a situation in which a line segment A0B0parallel to the z-axis is perspective-projected. First, the perspectiveprojection unit 5106 calculates a line segment A1B1 by performing aperspective transform based on a vanishing point C. The vanishing pointC is specified from among the 256 vanishing points by the programexecution unit 5101 or the input unit 5103. Then the perspectiveprojection unit 5106 calculates a final line segment A2B2 by performinga projection conversion in which a parallel projection of the linesegment to the xy plane takes place.

The frame memory 5107 is video RAM for storing one frame of image datadisplayed by the screen of the image display unit 5108.

The image display unit 5108 includes an image display control circuit,and a CRT or similar. The image display unit 5108 reads the image datastored in the frame memory 5107 and displays it on the CRT.

Operation

The following is an explanation of the operation of the multiple windowdisplay device of the present invention, constructed as described above.

FIG. 4 is a flowchart showing the operation procedure for the multiplewindow display device in the present embodiment.

FIG. 5 shows windows displayed on the image display unit 5108 by thisoperation.

First, the program execution unit 5101 activates and executes anapplication program and stores the display data (code data and imagedata) for windows generated by this application program in the storageunit 5201 (step S5120, S5121).

Next, the 3D position calculating unit 5104 calculates the position(coordinates for the four vertices of each window) within the 3D spacefor one of the windows stored in the storage unit 5201 and stores theresult (step S5122). This operation is carried out according to anotification from the program execution unit 5101 or the input unit5103. The coordinate system used here is the one shown in FIG. 3C.

Following this, when a new position has been calculated by the 3Dposition calculating unit 5104, the texture mapping unit 5105 readsimage data for the window from the storage unit 5102. The texturemapping unit 5105 then performs texture mapping for placing and pastingthe image data on a window whose size is determined by the four verticescalculated by the 3D position calculating unit 5104. This enablestexture data expressing one window to be generated (step S5123).

Then the perspective projection unit 5106 specifies a vanishing pointbased on a notification from the program execution unit 5101 or theinput unit 5103 and performs perspective projection, using thisvanishing point, on the texture data generated by the texture mappingunit 5105. The obtained perspective data is written in a correspondingposition in the frame memory 5107 (step S5124). If the vanishing pointis not indicated by a clear instruction from the program execution unit5101 or the input unit 5103, the perspective projection unit 5106performs perspective projection using a vanishing point whose x and ycoordinates coincide with the center of the window and whose zcoordinate is equivalent to the depth of the display screen.

Finally, the image display unit 5108 reads the perspective data (imagedata) written in the frame memory 5107 and displays it on the CRT (stepS5125).

Here, if a notification is made by the program execution unit 5101 orthe input unit 5103 to the 3D position calculating unit 5104,instructing it to change the position of a window stored in the storageunit 5102, the image display unit 5108 changes the display content byrepeating the above steps S5122 to S5125.

As explained above, the multiple window display device specifiescoordinates for four vertices of a window located in virtual 3D spaceand perspective-projects the window onto the display screen. As aresult, when the windows generated by the application program arepositioned parallel to the display screen in the 3D space, they aredisplayed on the display screen as in a conventional windowing system,as rectangles shown by windows 5130 and 5131 of FIG. 5. On the otherhand, when windows are positioned so that they are not parallel to thedisplay screen, they are displayed as trapezoids, shown by windows 5132to 5134 in FIG. 5.

In other words, the two windows 5130 and 5131 on the left side of thescreen in FIG. 5 are examples of windows that face the front, the window5132 in the top-right hand corner of the screen is an example of awindow whose upper edge is used as a rotation axis, with the lower edgehaving been inclined away from the user in the depth direction and thetwo windows 5134 and 5133 in the bottom-left of the screen are examplesof windows whose left edge is used as a rotation axis, with the rightedge having been inclined away from the user in the depth direction.

As a result of windows being displayed in perspective by the multiplewindow display device, the user has some difficulty in distinguishingcharacters and the like on the parts of the inclined windows positionedtowards the back of the 3D space, but should easily be able todistinguish the content of the parts of the windows near the front. Thedisplay area required by an inclined (perspective) window is less thanthat for displaying a window facing the front and the screen displayarea can thus be used effectively.

The screen display area is limited in notebook computers intended forportable use, and mobile information terminals such as electronicorganizers and PDAs (Personal Digital Assistants). In this case,effective use of the screen could be achieved by treating the displayedobjects so that a window on which an operation is being performed isdisplayed facing the front and, if a plurality of windows are beingviewed, windows on which an operation is not being performed aredisplayed in perspective. This invention may also be effective fordesktop computers, if a user is browsing various homepages on theInternet, or activating a variety of application programs.

In the present embodiment, a rectangular window is rotated using oneedge of the window as an axis, but the window may also be rotated usingtwo edges in succession as a rotation axis.

In the present embodiment, perspective projection was implemented aftertexture mapping had been performed on the entire display content of awindow. This order may be reversed, however, so that perspectiveprojection is only performed in advance on the window frame, enabling itto be projected onto the 2D display screen. This window frame is formedfrom four vertices for a window positioned in the 3D space. Then texturemapping may be performed by pasting the image data for the window storedin the storage unit 5201 into the projected window frame whileimplementing an linear transform such as an Affine transform. Byswitching the order of the texture mapping and the perspectiveprojection and performing a 2D linear transform when texture mappingtakes place, texturing may not be as precise, but performing 2D (x, y)calculation instead of 3D calculation (x, y, z) allows calculation forthe z-axis to be omitted and thus reduces the calculation load.

Furthermore, in the multiple window display device of the presentembodiment, the 3D position calculating unit 5104, the texture mappingunit 5105 and the perspective projection unit 5106 are separate from theprogram execution unit 5101, but a structure in which the programexecution unit 5101 executes all or part of the processing performed bythese units may be used.

The storage unit 5201 and the frame memory 5107 may share the samememory device.

Second Embodiment

The following is an explanation of a multiple window display device inthe second embodiment.

This multiple window display device is characterized by the ability todisplay windows in perspective so that a key part of the window displaycontent appears near the front of the 3D space.

Structure

FIG. 6 is a block diagram showing a structure for the multiple windowdisplay device in the second embodiment.

This multiple window display device includes a window analyzing unit5201, in addition to the structural components 5101 to 5108 of the firstembodiment. The following is an explanation of those points which differfrom the first embodiment.

An application program is activated by the program execution unit 5101and the display data (code data and image data) for one window stored inthe storage unit 5102. Once these operations have been completed, thewindow analyzing unit 5201 reads the code data for the window, specifiesthe key part of the data and determines the position of the window sothat this part of the data is displayed near the front. The windowanalyzing unit 5201 then notifies the 3D position calculating unit 5104of this position.

To be precise, the window analyzing unit 5201 notifies the 3D positioncalculating unit 5104 to take the edge of the window nearest to thespecified important part as the rotation axis and rotate the oppositeedge 45 degrees in the depth direction. The ‘key part’ is a part of thewindow that enables the user to instantly grasp the type and displaycontent. The actual criteria used for deciding on the key part follows apre-determined procedure described later in this document.

In this embodiment the 3D position calculating unit 5104 calculates thenew position having received notification of initial and displacementpositions for windows from the window analyzing unit 5201 as well asfrom the program execution unit 5101 and the input unit 5103.

Operation

The following is an explanation of the operation of the multiple windowdisplay device of the present invention, structured as described above.

The operation of the multiple window display device differs from theoperation performed in the first embodiment in that additionalprocessing for specifying the key part of the window is inserted betweenthe generation of the window display data (FIG. 4, step S5121) and the3D position calculation (FIG. 4, step S5122). This additional processingis explained below.

FIG. 7 is a flowchart showing the procedure performed when the windowanalyzing unit 5201 specifies the key part of the window.

FIG. 8 shows an example of a display screen produced when a key part hasbeen specified by the window analyzing unit 5201.

An application program is activated by the program execution unit 5101and display data (code data and image data) for one window is stored inthe storage unit 5201 (FIG. 4, step S5121). Then, the window analyzingunit 5201 reads the code data for the window (step S5220).

The window analyzing unit 5201 determines whether the code data includesa title displayed in the title bar of the window (step S5221). If thedata includes a title the window analyzing unit 5201 determines that thetitle bar is the key part of the window and notifies the 3D positioncalculating unit 5104 that the upper edge of the window is to be used asthe rotation axis (step S5222, FIG. 4, step 5122). The resulting screendisplay is the same as a window 5230 in FIG. 8. A title 5234 is includedin title bar 5235, so the window is displayed in perspective with theupper edge appearing near the front. The title is detected bydetermining whether a reserved word indicating the title of the windowis included in the code data.

If the window has no title, the window analyzing unit 5201 thendetermines whether text written horizontally from left to right isincluded in the code data (step S5223). If such text is included in thecode data, the window analyzing unit 5201 determines that the left sideof the window is the key part, and informs the 3D position calculatingunit 5104 that the left edge of the window is to be used as the rotationaxis (step S5224, FIG. 4, step S5122). The resulting screen display isthe same as a window 5231 in FIG. 8. The window is displayed inperspective, with the left edge, from which the text starts, near thefront of the 3D space.

If the code of data does not include horizontal text, the windowanalyzing unit 5201 then determines whether it includes text writtenvertically in lines ordered from right to left (step S5225). If the codedata includes vertical text the window analyzing unit 5201 determinesthat the right side of the window is the key part, and informs the 3Dposition calculating unit 5104 that right edge is to be used as therotation axis (step S5226, FIG. 4, step S5122). The window is displayedin perspective, so that the right edge, from which the text starts,appears near the front of the 3D space. Horizontal and vertical text aredetected by determining whether a reserved word specifying either ofthese text styles is included in the code data.

If the code data does not include vertical text either, the windowanalyzing unit 5201 then determines whether the code data includes animage (data in a predetermined image format) (step S5227). If an imageis included, the window analyzing unit 5201 determines it to be the keypart, specifies the edge of the window nearest to the position of theimage (the central point of the rectangular area surrounding the image)and informs the 3D position calculating unit 5104 that this edge is tobe used as the rotation axis (step S5228, FIG. 4, step S5122). Theresulting screen display is the same as windows 5232 and 5233 in FIG. 8.The window is displayed so that the right edge, which is closest to theimage, appears near the front of the 3D space.

If an image is not included either, the window analyzing unit 5201, as adefault process, informs the 3D position calculating unit 5104 that theleft edge of the window is to be used as a rotation axis. The resultingscreen display is the same as the window 5231 in FIG. 8. This is due tothe fact that significant display content is commonly found in the upperleft part of a window.

The multiple window display device specifies key parts of windows anddisplays the windows in perspective with the key parts appearing to benear the front, as explained above. This reduces the problems related todistinguishing display content when windows are displayed inperspective.

In the present embodiment, the window analyzing unit 5201 specified thekey part of a window according to the existence of a title, text style,and the existence of an image, but in addition to these criteria, thepart of a window in which display information is concentrated or thepart of the window in which a keyword representative of the page contentis positioned may also be specified as the key part.

If the entire display content of a window is an image, the windowanalyzing unit 5201 makes judgements according to the composition of thegraphics. Should the window be composed of an image of a human figurefor example, the window analyzing unit 5201 determines the position ofthe face of the figure by the distribution of flesh tones in the image,and places the window in perspective so that the face appears near thefront and is thus easy to see. Should the window be composed of an imageincluding a mountain, the window analyzing unit 5201 analyzes thedistribution of the green and brown of the mountain and the blue of thesky. If the mountain is taken as the key part, the window analyzing unit5201 puts the window in perspective so that the area in which the greenand brown of the mountain are common appears to be near the front. Thiskind of display information can be easily determined from image data,character codes and the like before they are developed into displaydata.

Third Embodiment

The following is an explanation of a multiple window display device inthe third embodiment.

This multiple window display device is characterized by the ability tobring a plurality of windows displayed scattered around the screen intoautomatic alignment.

Structure

FIG. 9 is a block diagram showing a structure for the multiple windowdisplay device in the third invention.

This multiple window display device shares the structural elements 5101to 5108 with the multiple window display device of the first embodiment,but differs in including an automatic aligning unit 5104 a inside the 3Dposition calculating unit 5104. The following is an explanation of thosepoints which differ from the first embodiment.

The automatic aligning unit 5104 a performs positional alignment on allof the windows (objects) stored in the storage unit 5201 using either afirst alignment method or a second alignment method. The secondalignment method involves window-stacking, but the first alignmentmethod does not.

In the first alignment method, the automatic aligning unit 5104 a storesin advance sixteen alignments for when the number of displayed windowsis one to sixteen, each alignment formed so that no window overlapsanother window. In the second alignment method, however, the automaticaligning unit 5104 a calculates positions for the windows so that eachwindow appears in perspective with its left edge appearing near thefront of the 3D space and its right edge inclined in the depthdirection. The windows are stacked horizontally an equal distance apart,with the right half of each window overlapped by the next window inline.

Then the automatic aligning unit 5104 a receives a notification from the3D position calculating unit 5104 specifying the alignment method andthe number of windows (objects). Next, the automatic aligning unit 5104a sends the position of each window in turn to the texture mapping unit5105.

Operation

The following is an explanation of the operation of a multiple windowdisplay device structured as described above.

This multiple window display device differs from the multiple windowdisplay device of the first embodiment in that the automatic aligningunit 5104 a calculates a new position during the 3D position calculationperformed in the first embodiment (step S5122). The following is anexplanation of the actual procedure for automatic alignment performed bythe automatic aligning unit 5104 a.

FIG. 10 is a flowchart showing the procedure for automatic alignmentperformed by the automatic aligning unit 5104 a.

FIGS. 11A and 11B show examples of display screens in which five windowshave been placed in automatic alignment by the automatic aligning unit5104 a. FIG. 11A shows an example of the first, alignment method, inwhich windows are not stacked, and FIG. 11B of the second alignmentmethod, in which windows are stacked.

First, the 3D position calculating unit 5104 receives an input regardingthe alignment method from the input unit 5103. The 3D positioncalculating unit 5104 refers to the storage unit 5201 and specifies anumber of windows n that are to be aligned. The 3D position calculatingunit 5104 then notifies the automatic aligning unit 5104 a of thealignment method and number of windows n (step S5230).

Upon receiving the notification, the automatic aligning unit 5104 adetermines the alignment method (step S5231). If the alignment method isthe first alignment method, in which windows are not stacked, theautomatic aligning unit 5104 a reads in sequence the pre-storedpositions to be used for the notified n windows (step S5232). Theautomatic aligning unit 5104 a notifies the texture mapping unit 5105 ofthe positions (step S5234). The resulting screen display is as shown inFIG. 11A.

In the case of the second display method, in which windows are stacked,the automatic aligning unit 5104 a calculates positions for each of nwindows in turn, in response to the notified number n. Positions arecalculated such that the windows are stacked horizontally an equaldistance apart with the left side of each window appearing near thefront and the right half overlapped by the next window in line (stepS5233). The automatic aligning unit 5104 a then informs the texturemapping unit 5105 of the positions (step S5234). The resulting screendisplay is shown in FIG. 11B.

As explained, a large number of windows displayed scattered around thescreen are automatically aligned in a perspective-projected state by themultiple window display device. This eliminates wasted area on thescreen and enables the position and content of all the windows to begrasped at a glance.

Here, once automatic alignment has been performed for a window, thealigned position can be used effectively as the default position foreach window. For example, the 3D position calculating unit 5104 maystore aligned positions for the windows obtained by automatic alignmentas default positions. If one of the windows in the automatically-alignedwindow group is specified by the input unit 5103, the 3D positioncalculating unit 5104 calculates the position of the window so that itfaces the front using the edge of the window near the front as arotation axis. A means for calculating a position enabling thefront-facing window to return to the default position at automaticalignment may also be added. This procedure is performed when a specificpart of the front-facing window (for example, a window adjustment buttonpositioned on the title bar) is indicated by the input unit 5103.

FIGS. 12A to 12C show an example of how the position of eachautomatically aligned window may be effectively used as its defaultposition.

FIG. 12A shows a display screen after automatic alignment has beenperformed, and corresponds to FIG. 11B.

FIG. 12B shows a display screen immediately after a window 5241 in FIG.12A has been selected by the input unit 5103.

Here, the 3D position calculating unit 5104 calculates a new position sothat the selected window 5241 is rotated to face the front with the edgethat was near the front of the 3D space in perspective (the left edge)remaining in a fixed position.

FIG. 12C shows a display screen immediately after the window adjustmentbutton 5245 positioned in the right corner of the title bar of thewindow 5241 has been pressed using the input unit 5103.

Here, the 3D position calculating unit 5104 reads the already storeddefault position for the window, on which the window adjustment button5245 has been pressed, as the new position for the window, and sendsthis position to the texture mapping unit 5105. This places the window5241 in perspective, returning it to the position following automaticalignment.

Thus, a window can be brought to face the front and returned to theautomatically aligned position at the touch of a button. This procedureis particularly useful in small portable information terminals, where itmakes windows easier to manipulate.

In the present embodiment, use of the second alignment method forstacking windows causes a plurality of windows to be aligned so that theright half of each window is covered by the next window. The addition ofthe window analyzing unit 5201 of the second embodiment, however, makesit easy to transform windows so that only insignificant parts of eachwindow are covered up.

In addition, instead of clicking a mouse to move a specified window tothe position immediately after automatic alignment or to the positionfacing the front, automatic interchanging of display positions may beperformed by moving a mouse cursor. In this case, a window on which themouse cursor is positioned is implicitly specified and is displayedfacing the front, while a window on which the mouse cursor is no longerpositioned is returned to its position immediately following automaticalignment.

Fourth Embodiment

The following is an explanation of a multiple window display device inthe fourth embodiment.

This multiple window display device is characterized by the ability tochange the position of the title and menu bars according to theorientation of the window.

FIG. 13 is a block diagram showing the structure of the multiple windowdisplay device in the fourth embodiment.

This multiple window display device includes a window information imageposition changing unit 5301 and a window information image orientationchanging unit 5302, in addition to the structural elements 5101 to 5108of the multiple window display device in the first embodiment. Thefollowing is an explanation of those points which differ from the firstembodiment.

When a window is displayed in perspective, the window information imageposition changing unit 5301 changes the position of the windowinformation image (title bar and menu bar image) so that it is displayedon the edge of the window near the front of the 3D space. To do so, thewindow information image position changing unit 5301 reads the positionof the window in the 3D space as calculated by the 3D positioncalculating unit 5104. If the window is in perspective with one edgenear the front of the 3D space, and the window information image is notdisplayed on that edge, the window information image position changingunit 5301 moves the window information image to the edge near the frontby overwriting the window image data stored in the storage unit 5102.

When the window is displayed in perspective and the window informationimage is displayed on the near edge, the window information imageorientation changing unit 5302 changes the image data so that only therectangle containing the window information image is rotated round sothat it faces the front (is parallel with the display screen). To do so,the window information image orientation changing unit 5302 divides theimage data for the whole window stored in the storage unit 5201 intodata for the window information image and data for the main part of thewindow. The window information image orientation changing unit 5302notifies the 3D position calculating unit 5104 that the windowinformation image is to be displayed facing the front and that theremaining main part of the window is to be displayed in perspective.

Operation

The following is an explanation of the operation of the multiple windowdisplay device of the present embodiment, structured as described above.

The multiple window display device of the present embodiment differsfrom the first embodiment in that processing of the window informationimage is added between calculation of the 3D position (FIG. 4, stepS5122) and the subsequent texture mapping (FIG. 4, step S5123) occurringin the first embodiment. This additional processing is explained here.

FIG. 14 is a flowchart showing the operating procedure for the windowinformation image position changing unit 5301 and the window informationimage orientation changing unit 5302.

When a position in the 3D space has been calculated by the 3D positioncalculating unit 5104 (FIG. 4, step S5122), the window information imageposition changing unit 5301 refers to this position to determine whetherthe window has been placed in perspective with one edge appearing nearthe front (step S5320).

If this is the case, the window information image position changing unit5301 next determines whether the window information image is located onthe near edge (step S5321). If the window information image is notdisplayed on the near edge, the window information image positionchanging unit 5301 rewrites the window image data stored in the storageunit 5201 so that the window information image is displayed on that edge(step S5322).

FIG. 15 shows an example of a display screen displaying two windows 5330and 5331. A window 5330 shows the situation before the position of thewindow information image is changed and a window 5331 the situationafter this change has taken place. Both windows were subject to texturemapping and perspective projection before being displayed.

It can be seen that the title bar and menu bar (window informationimage), which would normally be displayed positioned towards the back ofthe 3D space, are moved so as to be displayed on the edge nearer to thefront of the 3D space.

Next, the window information image orientation changing unit 5302determines whether an input instructing the window information image tobe placed facing the front has been received from the input unit 5103(step S5323). If such an input has been received, the window informationimage orientation changing unit 5302 specifies the key part from datastored in the storage unit 5201 by splitting data for the windowinformation image from data for the main part of the window (stepS5324). The window information image orientation changing unit 5302 alsoinstructs the 3D position calculating unit 5104 to calculate separatepositions for the two parts of the window, so that the windowinformation image faces the front with the main part of the window nextto it in perspective. (step S5325).

Following this, texture mapping and perspective projection are performedin turn on the window information image and the main part of the windowand the resulting image displayed on the screen (FIG. 4, steps S5123 toS5125).

FIG. 16 shows an example of a display screen, displaying a window 5332showing the situation before the orientation of the window informationimage is changed, and a window 5333 showing the situation after thewindow information image has been changed to face the front.

It can be seen that the title bar and menu bar (window informationimage) of the window in perspective have been rotated forward so thatthis part of the window faces the front.

As explained above, the multiple window display device of the presentembodiment can display the window information image for performingwindow operations and menu operations so that it always faces the front,even if a window is displayed in perspective. Alternatively, the windowinformation image may be fixed in place so that it always faces thefront. This prevents any difficulty in performing window operations andmenu operations when the window being displayed is in perspective.

In the present invention, processing was implemented so that the wholeof a planar window was folded into two parts: the window informationimage and the main part of the window. However, a method in which thewindow itself is presumed to be a 3D object and, when the window is inperspective, the title bar and the menu bar are displayed on a sidesurface of the object that is adjacent to the near edge of the windowsurface, may also be used. This enables data for the window informationimage and the main part of the window to be treated as a whole withoutneeding to be divided, making the position change and orientation changeprocessing simpler.

This method makes windows more visually appealing and simplifies windowoperations. It also eliminates the need for the window information imageto be displayed on the surface of the window, enabling the windowsurface to be utilized more effectively. Additionally, if windowoperations are capable of changing the orientation of a window displayedas a 3D object, the title bar and menu bar may be displayed in advanceon all four side surfaces adjacent to the window surface.

Fifth Embodiment

The following is an explanation of a multiple window display device inthe fifth embodiment.

The multiple window display device is characterized by the ability tocontrol the display of a scroll bar in accordance with the size of anunreadable display area created when a window is placed in perspective.

Operation

FIG. 17 is a block diagram showing a structure for the multiple windowdisplay device in the fifth embodiment.

The multiple window display device includes a scroll button positiondetermining unit 5401, a scroll bar attaching unit 5402 and anunreadable area determining unit 5403, in addition to the structuralelements 5101 to 5108 of the multiple window display device in the firstembodiment. The following is an explanation of those points that differfrom the first embodiment.

The unreadable area determining unit 5403 specifies an area that isunreadable after the window has been placed in perspective. To do so,the unreadable area determining unit 5403 calculates in advance thefinal display size of characters located at representative points in thewindow. Processing is then performed by the texture mapping unit 5105and the perspective projection unit 5106. The area where characterswould be displayed at a size smaller than 5 point×5 point is determinedto be the unreadable area. This calculation is performed with referenceto the size of characters in the window stored in the storage unit 5102,the window position calculated by the 3D position calculating unit 5104,and the transform constants (coordinates for vanishing points and theviewpoint) in the perspective projection unit 5106.

When an unreadable area is detected by the unreadable area determiningunit 5403, the scroll button position determining unit 5401 determinesthe position of a scroll button so that it reflects this. To do so, anedge running in the depth direction is selected as the scroll direction.The position of the scroll button is determined from the ratio of thelength of the readable area in the scroll direction to the length of theunreadable area in the scroll direction. The scroll button is positionedas if the unreadable area was not displayed in the window.

The scroll bar attaching unit 5402 attaches a scroll bar image to anedge of the window running in the depth direction. A scroll button isplaced on the scroll bar image at the position determined by the scrollbutton position determining unit 5401. The scroll bar attaching unit5402 then has the scroll bar image displayed in this way, by attachingthe scroll-bar image to the window image data stored in the storage unit5102.

Operation

The following is an explanation of the operation of a multiple windowdisplay device in the present embodiment, structured as described above.

The multiple window display device includes image processing for thescroll bar in addition to the operations in the first embodiment. Thisprocessing is included between the 3D position calculation (FIG. 4, stepS5122) and the subsequent texture mapping (FIG. 4, step S5123). Thefollowing is an explanation of this additional processing.

FIG. 18 is a flowchart showing the operating procedure for the scrollbutton position determining unit 5401, the scroll bar attaching unit5402 and the unreadable area determining unit 5403.

When a position is calculated for a window in the 3D space by the 3Dposition calculating unit 5104 (FIG. 4, step S5122), the unreadable areadetermining unit 5403 determines whether an unreadable area is presentin the window finally displayed following placement of the window inperspective (step S5420). If an unreadable area is present, theunreadable area determining unit 5403 specifies this area.

When an unreadable area has been found, the scroll button positiondetermining unit 5401 next determines the position of a scroll buttonfrom the size of this area. In other words, the position of the scrollbutton is determined as if only the readable area was displayed in thewindow.

Next, the scroll bar attaching unit 5402 attaches a scroll bar image tothe image data stored in the storage unit 5402. A scroll button isplaced on this scroll bar in the position determined by the scrollbutton position determining unit 5401 (step S5422).

The contents of the storage unit 5201 are thus rewritten so that theimage data is for a new window having a scroll bar. Next, texturemapping and perspective projection are performed on the new image data,and the result displayed on the screen, as in the first embodiment (FIG.4, step S5123 to S5125).

If the scroll button is then dragged using the input unit 5103, aconventional scrolling operation is performed by the program executionunit 5101. In other words, the position of the window itself does notchange, so that the scroll bar is still displayed at a position thatdeviates only by the size of unreadable area.

FIG. 19A is an example of a conventional window display, illustratingthe general concept for a scroll button 5430 and scroll bar 5431.

The position of the scroll button on the scroll bar shows the relativeposition of the current display content of the window to the displayobject as a whole.

FIG. 19B is an example of a screen display in which the lower half ofthe display area for a window has been determined to be an unreadablearea 5432.

Here, the scroll button 5430 is displayed at a position which treats thelower half of the display area 5432 as if it was not visible in thewindow. Accordingly, when the scroll button 5430 is moved downwards bythe input unit 5103, part of the area that was unreadable moves into areadable position.

As explained above, even if an unreadable area is created by displayingthe window in perspective, the multiple window display device in thepresent embodiment attaches a scroll bar for scrolling the content ofthe window towards the front of the 3D space in the depth direction. Asa result, the entire content of the display object can be distinguishedby dragging the scroll button in a simple operation, without requiringthe window itself to be moved.

In the present embodiment, a scroll bar is attached when an unreadablearea is detected. However, a structure in which a scroll bar is attachedwhen part of a document cannot be displayed in the window may also beused, regardless of whether an unreadable area exists or not. In thiscase, the scroll button signifies a position at which the area treatedas not visible on screen is obtained by adding the unreadable area tothe area extending beyond the bottom of the window. Additionally,operations can be made more convenient for the user by color coding thescroll bar in the texture mapping process to differentiate between theunreadable area and the area extending beyond the bottom of the window.

Sixth Embodiment

The following is an explanation of a multiple window display device inthe sixth embodiment.

The multiple window display device of the present embodiment ischaracterized by the ability to display windows in layers, with eachlayer distinct from other layers and in perspective.

Structure

FIG. 20 is a block diagram showing a structure for the multiple windowdisplay device in the sixth embodiment.

The multiple window display device of the present embodiment includes awindow layer separating unit 5501 and a window layer generating unit5502, in addition to the structural elements 5101 to 5108 of themultiple window display device in the first embodiment. The following isan explanation of the differences from the first embodiment.

The window layer separating unit 5501 detects a window with a layeredconstruction from the windows stored in the storage unit 5102, andseparates that window into layers.

Some application programs which handle a mixture of text and images or aplurality of mixed images use a layered construction, in which imagesand text are provided on separate layers, enabling editing andoperations to be performed independently for each layer. This is theformat used in graphics design software and document layout softwareused for desktop publishing. When this kind of application program isexecuted by the program execution unit 5101, code data and image datafor a window having a layered construction are stored in the storageunit 5102.

The window layer separating unit 5501 detects layered windows byanalyzing the code data of each window stored in the storage unit 5102.The window layer separating unit 5501 then fetches the image data forall the layers in a multilayer window apart from the base (bottom)layer.

The window layer generating unit 5502 generates independent windowlayers from the layers separated by the window layer separating unit5501. In other words, the window layer generating unit 5502 generatesnew image data so that the image data for each of the layers fetched bythe window layer separating unit 5501 becomes a separate window, andstores the generated data in the storage unit 5102.

Operation

The following is an explanation of the operation of the multiple windowdisplay device of the present embodiment, structured as described above.

The operation of the multiple window display device differs from thefirst embodiment by including window layer processing between thegeneration of code data and image data for the window (FIG. 4, stepS5121) and the subsequent calculation of a 3D position (FIG. 4, stepS5122). The following is an explanation of this additional processing.

FIG. 21 is a flowchart showing the operating procedure for the windowlayer separating unit 5501 and the window layer generating unit 5502.

When display data (code data and image data) for a window with a layeredconstruction has been stored in the storage unit 5201 by the programexecution unit 5101 (FIG. 4, step 5121), the window layer separatingunit 5501 specifies the layered window out of the code data stored inthe storage unit 5102. The window layer separating unit 5501 divides theimage data into layers and then fetches the image data for all of thelayers apart from a base layer (step S5520).

FIG. 22A illustrates the concept of layer separation when there are twolayers (step S5521). A composite window 5530 stored in the storage unit5201 is shown divided into a base layer 5531 and an upper layer 5532.

Next, the window layer generating unit 5502 generates image data for theupper layer 5532 fetched by the window layer separating unit 5501 sothat it forms a separate window and stores the data in the storage unit5201 (step S5521). Then, the layer window generating unit 5502 notifiesthe 3D position calculating unit 5104 of the newly generated windowlayer 5532. The notification stipulates that the window layer 5532should be parallel to the base layer 5531 in the 3D space, and aspecified distance nearer to the front of the 3D spade (step S5522).

Following this, the 3D position calculating unit 5104 calculates aposition for the base layer window 5531 as specified by the programexecution unit 5101 or the input unit 5103. The 3D position calculatingunit 5104 also calculates a position for the upper layer window 5532 asspecified by the window layer generating unit 5502. The position of theupper layer window 5532 is calculated so that it is located at aposition where the base layer window 5531 would be were it moved aspecified distance forward perpendicular to its plane (FIG. 4, stepS5122).

FIG. 22B shows an example of a screen display when a layered window isplaced in perspective.

The upper layer 5532 is displayed so that it seems to float over thebase layer 5531.

The multiple window display device in the present embodiment displays awindow with a layered construction in perspective, and separated intolayers. As a result, information which could not be judged from aconventional ‘flat’ screen, such as which objects are located in whichlayer, can be grasped at a glance. As all the layers can be displayedsimultaneously, objects on any layer can be specified immediatelywithout needing to switch between layers. This enables editingoperations and the like spanning a plurality of layers to be performedsmoothly.

In the present embodiment, the window layers are transparent and thewhole of each overlapping window layer is displayed. If a window layerselected by the user is made semi-transparent, however, it can behighlighted, making it easier to see.

Seventh Embodiment

The following is an explanation of a multiple window display device inthe seventh embodiment.

The multiple window display device of the present embodiment ischaracterized by the ability to separate the result of a keyword searchfrom an original window as a window layer, displaying the new layer sothat it stands out from the original window.

Structure

FIG. 23 is a block diagram showing a structure for the multiple windowdisplay device in the seventh embodiment.

The multiple window display device of the seventh embodiment includes akeyword search unit 5601, a search result output unit 5602 and a windowlayer generating unit 5603 in addition to the structural elements 5101to 5108 of the first embodiment. The following is an explanation of thedifferences from the first embodiment.

The keyword search unit 5601 searches for a keyword in a documentdisplayed in a window. This means that the keyword search unit 5601searches among the code data of the document stored in the storage unit5201 for a character string identical to a keyword indicated by theinput unit 5103. If an identical character string is found, the keywordsearch unit 5601 informs the window layer generating unit 5603 of thewindow, the position(s) pinpointed by the search and the keyword.

The window layer generating unit 5603 generates a new window layer, inwhich only the keyword searched for is arranged at the positions foundin the search, according to the notification received from the keywordsearch unit 5601. This means that only image data including the searchedcharacter string is taken from the image data of the window indicated bythe keyword search unit 5601. The window layer generating unit 5603stores this image data in the storage unit 5201 as the image data for anew window layer.

The search result output unit 5602 informs the 3D position calculatingunit 5104 of the position of the window layer generated by the windowlayer generating unit 5603. This means that the search result outputunit 5602 specifies the position of a new window layer so that thepositional relation between the search object, in other words theoriginal window, and the newly generated window is the same as thepositional relation between the base layer 5531 and the upper layer 5532in the sixth embodiment.

Operation

The following is an explanation of the operation of the multiple windowdisplay device in the present embodiment, structured as described above.

The operation of the multiple window display device of the presentembodiment differs from the first embodiment in that additional windowlayer processing is included between the generation of code data andimage data for the window (FIG. 4, step S5121) and the subsequentcalculation of a 3D position (FIG. 4, step S5122). The following is anexplanation of the differences from the first embodiment.

FIG. 24 is a flowchart showing the operating procedure for the keywordsearch unit 5601, the search result output unit 5602 and the windowlayer generating unit 5603.

When a keyword is input by a user via the input unit 5103. (step S5620),the keyword search unit 5601 searches for a character string identicalto this keyword from among the code data for the entire document storedin the storage unit 5201 (step S5621).

If an identical character string is found (step S5622), the keywordsearch unit 5601 informs the window layer generating unit 5603 of foundpositions and keywords for each window.

The window layer generating unit 5603 generates image data for a windowlayer consisting only of instances of the character string found duringthe search according to this information, and stores the image data inthe storage unit 5201 (step S5623).

Then, the search result output unit 5602 directs the 3D positioncalculating unit 5104 to calculate 3D positions so that the windowgenerated by the window layer generating unit 5603 appears to float overthe original window (step S5624).

Consequently, the 3D position calculating unit 5104 calculates theposition for the search object, that is the original window, asspecified by the program execution unit 5101 or the input unit 5103. The3D position calculating unit 5104 also calculates the position for thewindow layer generated by the window layer generating unit 5603 based onan indication from the search result output unit 5602. This position iscalculated so that the window layer is positioned in parallel with, andonly a specified distance away from the original window (FIG. 4, stepS5122).

FIG. 25 shows the above procedure and the final display screen.

If a keyword ‘window’ is included in the search object, that is theoriginal window 5630, a new window layer 5631 is generated by the windowlayer generating unit 5603, and displayed so that it appears to standout from the original window 5630.

An original window and a window showing a keyword search result can bedisplayed in perspective in this way, enabling the search result to bediscerned at a glance. In a conventional ‘flat’ screen, processing suchas representing the search result in a different color was required toachieve this, but the multiple window display device of the presentembodiment requires no such special processing.

A device capable of handling a plurality of keywords can easily beachieved by having each keyword displayed on a differentsemi-transparent layer using a different color.

Eighth Embodiment

The following is an explanation of a multiple window display device inthe eighth embodiment.

The multiple window display device of the present embodiment ischaracterized by the ability to display a plurality of inter-relatedwindows linked and in perspective.

Structure

FIG. 26 is a block diagram showing a structure for the multiple windowdisplay device in the eighth embodiment.

The multiple window display device of the present embodiment includes awindow linking unit 5701, a relative window position storage unit 5702and a relative window position changing unit 5703, in addition to thestructural elements 5101 to 5108 of the first embodiment. The followingis an explanation of the differences from the first embodiment.

The window linking unit 5701 specifies a group of inter-related windowsfrom a plurality of windows currently displayed scattered around thescreen. In the present embodiment, inter-related windows are windowsgenerated by the same application program. These windows may, forexample, correspond to the situation when a plurality of pages belongingto the same document are displayed simultaneously on a plurality ofwindows, as in a word-processing program or similar application program.

To achieve this, the window linking unit 5701 specifies windowsbelonging to the same application program from among a plurality ofwindows stored in the storage unit 5102. Windows are specified based ona notification from the program execution unit 5101, which is executingan application program. The window linking unit assigns an order to thewindows by analyzing the code data for each window, and then-notifiesthe relative position storage unit 5702 of the order. Windows may bearranged, for example, in ascending order of page number.

The relative position storage unit 5702 generates and stores linkinginformation for calculating the 3D position of each window in a group ofwindows that have been specified and ordered by the window linking unit5701. Positions are calculated so that the windows are connectedhorizontally, and arranged as a concertina. This means that the relativeposition storage unit 5702 stores the relative coordinates for theposition of a first window placed in perspective and the next connectedwindow. The initial position is predetermined, and each window is placedin perspective, using the near edge as a rotation axis and rotating theopposite edge 45°.

The relative position changing unit 5703 deletes only a specified windowfrom the concertina-like sequence of windows on the display screen byfolding it up. This means that the relative position changing unit 5703changes the storage content of the relative position storage unit 5702so that the windows are connected together bypassing a window specifiedby the input unit 5103.

Operation

The following is an explanation of the operation of the multiple windowdisplay device in the present embodiment, structured as described above.

The operation of the multiple window display device in the presentembodiment differs from the first embodiment in that window linkingprocessing is added between the generation of the code data and theimage data for the window (FIG. 4, step S5121) and the subsequentcalculation of a 3D position (FIG. 4, step S5122). The following is anexplanation of this additional processing.

FIG. 27 is a flowchart showing operating procedures for the windowconnecting unit 5701, the relative position storage unit 5702 and therelative position changing unit 5703.

Assume that the program execution unit 5101 has already stored thedisplay data (code data and image data) for five windows displayingpages 1 to 5 of a document based on the execution of the sameapplication program in the storage unit 5102, and notified the windowconnecting unit 5701 of this. (FIG. 4, step S5121).

The window connecting unit 5701 specifies the five windows stored in thestorage unit 5201 according to the notification, and notifies therelative position storage unit 5702 of the five pages in ascending orderof page number by referring to the code data for the windows (stepS5720).

The relative position storage unit 5702 generates and stores initiallinking information for the windows of which it was informed by thewindow connecting unit 5701. The information is generated so that thewindows will be connected in the notified order arranged as a concertina(step S5721).

FIG. 28A shows a screen display when the initial linking information isunchanged (step S5722) and the position of the windows for the pages 1to 5 is calculated by the 3D position calculating unit 5104 and windowsdisplayed, according to the link information (steps S5122 to S5125).

The five windows are displayed in perspective, lined up horizontally,with neighboring windows sharing a common edge. Windows inclinealternately to the left and right in the depth direction.

If the relative position changing unit 5703 receives a notification fromthe input unit 5103 instructing it to fold up a specific window (stepS5722), the relative position changing unit 5703 changes the storagecontent (link information) in the relative position storage unit 5702 sothat the windows are connected bypassing the specified window (stepS5723).

FIG. 28B shows an example of a display screen when the link informationis changed so that the window for page 3 is folded up (steps S5722,S5723), and positions for the windows of the following pages 4 and 5 arecalculated by the 3D position calculating unit 5104 and displayed,according to the link information (FIG. 4, step S5122 to S5125).

The position of pages 4 and 5 is changed so that they are connected topage 2. The relative position of page 4 to page 3 is saved.

The multiple window display device of the present embodiment displaysinter-related page windows in perspective, with the edge of one windowconnected to the edge of the next window, enabling the user to grasp therelationship between windows at a glance by noting the way in which theyare connected.

Application programs such as word-processing programs often handledocuments composed on a plurality of pages. In this case, it isdifficult to display a plurality of pages simultaneously in aconventional windowing system in which windows face the front. Windowshave to be displayed overlapping other windows, or only a few pages canbe displayed. In the multiple window display device of the presentinvention, a large number of pages can be and displayed in sequence in alimited display area without overlapping.

Additionally, the user is able to select which windows are to bedisplayed, and windows that need not be displayed are stored folded upbehind other windows. This enables inter-relations between windows, suchas in a sequence of pages, to be preserved, while only a plurality ofsignificant windows are displayed at any one time.

Furthermore, as the bit-map pasting performed by the texture mappingunit 5105 is not performed on the windows which have been folded upbehind other windows, one part of the texture-mapping processing can beomitted.

In the present embodiment, inter-related windows were described aswindows generated by the same application program. Frequently, however,pages that are from different documents but belong to the same projecthave a strong inter-relationship. Thus, specifying inter-related windowsin advance enables even windows generated by different applicationprograms to be displayed connected to each other.

A means for performing editing techniques such as copying or moving anobject between windows displayed connected in sequence may also beprovided.

FIGS. 29A and 29B show examples of screen displays when a means forperforming operations spanning connected page windows on an object isprovided.

FIG. 29A shows a screen display when the windows of pages 1 to 5 aredisplayed based on initial link information. This drawing is identicalto FIG. 28A.

FIG. 29B shows a situation in which page 2 and 3 are folded up so thatpage 1 and page 4 are displayed as facing pages, and an object is copiedfrom page 1 to page 4. This is realized by overwriting display data(code data and image data) for page 1, stored in the storage unit 5102,on page 5, so that an object indicated on the original window by theinput unit 5103 is copied onto the target window.

Ninth Embodiment

The following is an explanation of a multiple window display device inthe ninth embodiment.

The multiple window display device of the present embodiment relates tospecial character display processing used when windows are displayed inperspective, and is characterized by not needing to perform aperspective projection transform (3D processing) directly on characters.Instead, a perspective projection transform is performed on windowframes and the obtained 2D data is used to display characters.

Structure

FIG. 30 is a block diagram of the multiple window display device in theninth embodiment. This multiple window display device includes a 3Dinformation management unit 100, a font database 110, a rotationprocessing unit 120, a perspective transform unit 130, a transformparameter calculating unit 140, a font transform unit 150, a fontreading unit 160, a display control unit 170, a display unit 180 and aninput unit 190.

FIG. 31 shows a situation in which a window W is placed in a virtualspace constructed within the multiple window display device. In thisdrawing, point E is the viewpoint and screen F is a display screen for aperspective transform view seen from the viewpoint E. The window W isarranged in the virtual space so that it appears to be to the rear ofthe screen F when viewed from the viewpoint E. The positions of areaswithin the virtual screen can be expressed by an XYZ coordinate systemin which a screen parallel to the display screen F is an XY screen. Forconvenience, the bottom left vertex f of the display screen F is takenas the coordinate origin (0,0,0). However, positions in the window W areexpressed by a fixed coordinate system (hereafter referred to as thewindow coordinate system) in which the bottom left vertex B of thewindow W is the coordinate origin. The axes in the window coordinatesystem are x, y, z (lower case). A character string 201 and a character202 are displayed in the window W.

The 3D information management unit 100 stores information relating tothe window W in the virtual space and the character string 201 andenlarged character 202 displayed in the window. The 3D informationmanagement unit 100 thus has a window information unit 101, a characterinformation unit 102 and a miscellaneous information unit 103.

The window information unit 101 stores window information, includingcoordinates showing the position of the window W in the virtual space,and window rotation angle information showing the number of degrees thewindow W is rotated around each of the x, y and z axes. This informationis stored for each window in the virtual space.

FIG. 32 shows example window information 300 stored in the windowinformation unit 101.

A window identifier showing which window in the virtual space theinformation refers to is stored in a window identifier column 301. Onlyone window is shown in the example in FIG. 31, but if a plurality ofwindows are arranged in the virtual space, windows can be distinguishedusing this identifier.

Coordinate values for the window coordinate origin (the left vertex ofthe window, point B in the example in FIG. 31) in the virtual space arestored in a spatial coordinates column 302.

Coordinate values for the four vertices of the window in the windowcoordinate system are stored in a window coordinate column 303. Here,coordinates in the window coordinate system for the vertices of thewindow W shown in FIG. 31, that is A, B, C, and D, are stored.

Rotation angles showing the number of degrees the window has rotatedabout each of the x, y and z axes are stored in a rotation angle column304. Here, the window W in FIG. 31 is arranged on a plane that includesthe x and y axes of the window coordinate system, so the angles are all0°.

The character information unit 102 controls character information, thatis information relating to characters displayed in windows. Informationcombining a character code representing the character, font size, fonttype and character coordinates showing the position of a character inthe virtual space are stored for each character. The actual characterimage itself is not stored, however. The character coordinates representone point, an upper left corner of a font character in the window.

FIG. 33 shows example character information 400 stored in the characterinformation unit 102. The content displayed here relates to thecharacter string 201 and the character 202 in the window W of FIG. 31.The character in the window W to which each group of data corresponds tois also shown.

A two-byte, hexadecimal display code indicating each displayed characteris stored in a character code display column 401. In the example of FIG.33, a code indicating the character ‘3’ at the beginning of the firstline of the character string 201 in FIG. 31 is stored in the first row.

Values showing the font size for each character in point units arestored in a font size column 402.

Codes showing font types are stored in a font type column 403. Here, acode ‘M’ indicating Mincho typeface is stored.

Window coordinate system coordinates for a vertex of the upper leftcorner of each font character are stored in a character coordinatecolumn 404. The content shown in FIG. 33 relates to characters in thewindow W arranged on a plane that includes the x and y axes (FIG. 31),so the z coordinate value for each character is 0. Also, as can be seenfrom FIG. 33, the y coordinate value for characters on the same line inwindow W is identical.

Only part of the information for constructing a window is stored in thewindow information unit 101 and the character information unit 102. Theother information, including image data for objects displayed in thewindow and the like, is stored in the miscellaneous information unit103.

Font data is stored in the font database 110, along with uniquecharacter codes indicating each font character. These character codesare the same as part of the character information stored in thecharacter information unit 102.

The display unit 180 is a computer display apparatus and displayswindows containing text.

The input unit 190 is an input device such as a keyboard or a mouse. Auser uses this device to give an input for window rotation. The inputunit 190 informs the display control unit 170 of the input.

The rotation processing unit 120, receives the content of a windowrotation input (rotation object window, rotation axis, rotation angle)made by the user using the input 190. The content is received via thedisplay control unit 170. The rotation processing unit 120, rotates awindow within the virtual space according to the input, and then updatesthe window information and character coordinates according to therotation result.

FIG. 34 shows the result of this rotation processing, a window W′. Theprocessed object is the window W in FIG. 31, and a rotation of 30degrees is performed around the left edge (y-axis) of the window W.

The rotation processing unit 120, reads the window information shown inFIG. 32 and the character information (coordinates) shown in FIG. 33from the 3D information management unit 100 and performs the followingprocessing.

First, processing is performed on the window information by changing thecoordinates for the vertices of the window W to the coordinatesfollowing rotation. Here, rotation takes place around the y-axis, so thecoordinates for the C and D vertices are moved by changing the values ofthe x and z coordinates for the two vertices. If the coordinate valuesfor the C and D vertices following rotation are expressed so that the xcoordinate is x1′ and the z coordinate z1′, these coordinate values canbe found using the following formula:

x1′ =x1 cos 30°

z1′ =x1 sin 30°

where value x1 is the x coordinate value for the C and D points beforerotation is performed.

FIG. 35 shows the content of window information for the window W afterrotation processing has been performed. Comparison with the content ofFIG. 32 reveals the change in content caused by the rotation processing.The parts of the information that have changed are shaded in thedrawing.

Next, processing is performed on the character information, changingposition coordinates for each character in the window W. The x and zcoordinate values are changed in the same way as the window coordinatevalues, but the size of the change differs according to the distance ofa character from the y-axis, in other words the x coordinate valuebefore the change. The values of both the x and z coordinates after thechange can be found using the same formula as was used to find thecoordinate values for the window vertices after the change.

xn′ =xn cos 30°

zn′ =xn sin 30°

Here, xn is the x coordinate value for a character position prior torotation, xn′ is the x coordinate value for a character positionfollowing rotation, and zn′ is the z coordinate value followingrotation.

FIG. 36 shows the content of character information for a windowfollowing rotation processing. Comparison with the content of FIG. 33allows the change caused by the rotation processing to be understood.The parts of the character information that have changed are shaded inthe drawing.

When the rotation processing unit 120, has completed its processing, theperspective transform unit 130 performs a perspective transform on thewindow in the virtual space and the character position information forcharacters in the window, taking the display of the display unit 180 asthe display screen. This perspective transform is performed based oninformation read from the 3D information management unit 100. Thisinformation is obtained after rotation processing and includesinformation on various coordinates and window angle information. Fromhereon, a window in the virtual space that has not yet beenperspective-transformed is generally referred to as a ‘3D window’, and aperspective-transformed window for display use as a ‘2D window’. Whenthere is no need to distinguish between the two types of windows,however, both are referred to simply as ‘windows’. The perspectivetransform method used is well-known in the art, and so is only explainedin brief here.

The perspective transform is performed by projecting a window onto thedisplay screen, while keeping the viewing angle constant, to obtain a 2Dwindow. The distance between each vertex of the 2D window is influencedby the z coordinate value for each vertex of the corresponding 3D windowthat is the origin for the transform. If the gap between the zcoordinate values for the two points is large, the distance between thetwo points on the display screen will be short, and the 2D window willbe in the shape of a trapezoid.

In FIG. 34, a window Wi′, which is produced by performing theperspective transform on the window W′ (window W following the rotationprocessing) is also displayed.

On completing its processing, and obtaining the processing result,. theperspective transform unit 130 sends information for the 2D window, suchas that shown in FIGS. 37A and 37B, to the display control unit 170.

FIG. 37A shows example window information for the 2D window resultingfrom the perspective transform, in other words the data for the windowWi′ obtained by performing the perspective transform on the window w′.Comparison with the data prior to perspective transform shown in FIG. 35allows changes in the content to be understood.

The 2D window information 800 is formed from a window identifier column301 and a window display coordinates column 801.

The window identifier column 301 has already been explained in thedescription for FIG. 32.

Coordinates for the vertices of the 2D window on the perspective screenformed following the perspective transform are stored in the windowdisplay coordinates column 801. These coordinates are coordinatesbelonging to a 2D coordinate system for perspective screen use.

FIG. 37B shows examples of information relating to characters in a 2Dwindow. This is the character information (in FIG. 34, a characterstring 511 and characters 512 and 513) produced as a result of aperspective transform performed on the character information shown inFIG. 36 (in FIG. 34, a character string 501 and characters 502 and 503).In the character information, only position coordinates are the objectof the perspective transform. When the perspective transform unit 130sends character information to the display control unit 170 followingthe perspective transform, it also sends the 3-dimensional coordinatesprior to perspective transform (values in the window coordinate system)for each character, in order to make things more convenient forsubsequent processing. Comparison with the data prior to perspectivetransform shown in FIG. 36 allows the changes in the content to beunderstood. FIG. 37B also shows the corresponding character in thewindow for each set of data.

Display character information 810 is formed from a display positioncolumn 811 and a 3D coordinate column 812.

2D coordinates showing the display positions for characters on thedisplay screen are shown in the display position column 811. Thesecoordinates belong to the 2D coordinate system and are for perspectivescreen use.

3D coordinates showing character positions prior to perspectivetransform (window coordinate system) are stored in the 3D coordinatecolumn 812.

The perspective transform unit 130 informs the transform parametercalculating unit 140 of the coordinate information for a 2D window, andrequests the calculation of transform parameters.

Upon receiving an input from the perspective transform unit 130, thetransform parameter calculating unit 140 calculates transform parametersfor a font character from the coordinate information received togetherwith the input, and notifies the font transform unit 150 of these. Thetransform parameters are expressions showing the content of thetransform performed when the font transform unit 150 transforms the fontcharacter, creating a character image to be displayed in the 2D window.

The following is an explanation of the procedure used by the transformparameter calculating unit 140 to calculate the transform parameters.

FIG. 38 shows a 2D window 900 produced when a perspective transform isperformed on the window W in FIG. 31.

FIG. 39 shows a 2D window 1000 produced when a perspective transform isperformed on the same window W as in FIG. 38 after it has been rotatedaround the y-axis in a 3D space.

A character string 901 is displayed in the window 900 and a characterstring 1002 in the window 1000. Enlarged character views 902 and 903 inFIG. 38 and 1002 and 1003 in FIG. 39 are enlargements of singlecharacters in the character string used to demonstrate the charactertransform.

In the drawings, A to D and E to H are coordinates for the windows, andP to S and T to W are coordinates for the four corners of the characterimages 902 and 1002.

The transform parameter calculating unit 140 stores 2D coordinates for awindow on which no rotation processing has been implemented, that is thecoordinates for A to D of window 900, as reference values. Coordinatesproduced after window rotation and a perspective transform have beenperformed on the window, like those in FIG. 37, are output from theperspective transform unit 130. Suppose that the square ABCDrepresenting the window 900 has been transformed two-dimensionally intothe square EFGH representing the window 600. In this case, the characteroutline shown by the square PQRS can be transformed into the characteroutline shown by the square TUVW using a similarity transform. Theexpressions used for this similarity transform are output to the fonttransform unit 150 as the transform parameters.

The processing performed by the font transform unit 150 takes place inthe following order. First, the size of the font for each character isdetermined by finding the font scale. Then a font character of thedetermined size is transformed according to the transform parameters.The font transform unit 150 stores a transform formula equivalent to a‘Z coordinate value/font scale characteristic curve’ shown in FIG. 40 inorder to find the font scale. The Z coordinate values substituted intothe transform formula are coordinate values belonging to a virtual spaceXYZ reference coordinate system. When a Z coordinate value is 0, thismeans that the coordinate is positioned in the perspective transformprojection plane (an xy plane in which a z coordinate value is 0). Whenthe Z coordinate increases, this means that the coordinate has movedfurther away from the projection plane. In other words, the font scaleof characters with position coordinates near the projection plane isclose to 1 and the font scale of characters with position coordinatesfar from the projection plane is close to 0.

In order to find the font scale, the font transform unit 150 obtainsinformation for each character from the character information unit 102via the font read unit 160. This information describes the z coordinatevalue of the character coordinates, and font size, and is taken from theinformation shown in FIG. 36. The z coordinate value (from the windowcoordinate system) is converted into a Z coordinate value belonging tothe virtual space coordinate system, and the obtained value issubstituted into the transform formula to find the font scale. The fontsize is then multiplied by the font scale to find the font size to beused for display on the display screen.

FIG. 41 shows font sizes found as described above, which are used todisplay each character in the window Wi′. In the drawing, columnsshowing the character in the window to which the data corresponds andcurrent font size have been included for each piece of data for ease ofreference.

The font transform unit 150 reads corresponding font characters for eachcharacter code from the font database 110 via the font reading unit 160,and uses the above display font size information to determine coordinatevalues for each font character. The font coordinates for the top leftcorner (P in the example in FIG. 38) are (0,0), and the coordinates forthe remaining three corners are determined by the display size value.For example, if the font size is a and scale is b, the display size isa×b and the coordinates for each corner are Q (0, −a×b), R (a×b, −a×b),S (a×b, 0).

The font transform unit 150 substitutes the font coordinate values foundin this way into the transform parameter formula sent from the transformparameter calculating unit 140, and produces a character image fordisplay.

Next, the font transform unit 150 sends the character image produced bytransforming the font to the display control unit 170, together with 3Dposition coordinates for the image.

The display control unit 170 combines the character image produced bythe font transform unit 150 with a 2D window, which has been transformedby the perspective transform unit 140. When doing so, the displaycontrol unit 170 determines the position of each character image in the2D window by referring to 2D character coordinates for both before andafter transformation, sent from the perspective transform unit 140, andto character position coordinates received from the font transform unit150 together with the character images. Suppose that the character image‘A’ is sent from the font transform unit 150 together with the positioncoordinates (x1, y1, z1). In this case, the display control unit 170finds the data whose coordinates prior to conversion are (x1, y1, z1)from among the character coordinates received from the perspectivetransform unit 140, and places the character image ‘A’ at the position(x1′, y1′) denoted by the corresponding post-transformation coordinates.

The result of arranging each character image in a specified position inthis way is shown in FIG. 39.

Operation

FIG. 42 is a flowchart showing the operation of rotation processing fora window including character font, in the ninth embodiment of the windowdisplay device in the present invention. The following is an explanationof the processing with reference to the flowchart.

First, a user uses the input unit 190 to indicate that window rotationis to be performed in a screen displayed on the display unit 180. Uponreceiving this input, the display control unit 170 sends information ona rotation axis and angle of rotation obtained by analyzing the contentof the input to the rotation processing unit 120, instructing it torotate a 3D window in virtual space based on this information (stepS1301).

Upon receiving this input, the rotation processing unit 120, implementsrotation processing on the 3D window. The information stored in the 3Dinformation management unit 100 prior to rotation is updated accordingly(step S1302.)

After the rotation processing unit 120, finishes the above processing,it informs the display control unit 170 that processing has beencompleted. Upon receiving this information, the display control unit 170instructs the perspective transform unit 130 to perform perspectivetransform processing. The perspective transform unit 130 performsperspective transform processing, producing the information for a 2Dwindow to be displayed on the display unit 180 (step S1303).

Upon finishing the above perspective transform processing, theperspective transform unit 130 sends coordinates for the transformed 2Dwindow to the transform parameter calculating unit 140, and instructsthe transform parameter calculating unit 140 to find the transformparameters. The transform parameters are to be used when the fonttransform unit 150 produces character images for display by performing asimilarity transform on the character fonts. The perspective transformunit 130 also sends the 2D window information and character coordinateinformation (after transformation) resulting from the perspectivetransform to the display control unit 170.

Upon receiving the input from the perspective transform unit 130, thetransform parameter calculating unit 140 calculates transform parametersfrom the coordinates for the 2D window after the perspective transformreceived together with the input, and outputs the calculated transformparameters to the font transform unit 150 (step S1304).

Upon receiving the transform parameters from the transform parametercalculating unit 140, the font transform unit 150 instructs the fontreading unit 160 to read a font character corresponding to one of thecharacters in the window. Upon receiving the input, the font readingunit 160 reads information for a character code, a font size, a fonttype and character position coordinates from the character informationunit 102 (step S1305). Following this, the font reading unit reads afont character that agrees with both the character code and the fonttype from the font database 110. Then, this font character, characterfont size and character coordinates are sent to the font transform unit150 (step S1306).

The font transform unit 150 produces a character image for display bytransforming the font based on the information sent from the fontreading unit 160, the transform parameters and the font scale. Thecharacter image thus produced is sent to the display control unit 170along with 3D character coordinate information read from the characterinformation unit 102 (step S1307).

The font transform unit 150 repeats the above processing until there areno unprocessed characters left in the character information unit 102.Once all characters have been processed, the font transform unit 150sends the display control unit 170 a notice to this effect (step S1308).

Upon receiving the notice from the font transform unit 150, the displaycontrol unit 170 combines the 2D window received from the perspectivetransform unit 130 and the character images received from the fonttransform unit 150, and displays the resulting image on the display unit180 (step S1309).

In the window display device in the present embodiment, as explainedabove, character images in a 3D window, produced by expanding 3D data ina conventional virtual space and then performing rotation andperspective transform processing on the expanded data, are separatedfrom other window information and transformed into pseudo-3D shapes.Then the transformed data is combined with a 2D window, which has beensubject to a perspective transform. This enables processing fortransforming a 3D window including character images to a 2D window anddisplaying the result to be performed faster than in the prior art. Inaddition, the character images need not be expanded in the virtual spaceas a set of 3D data, so that the amount of data for the 3D window isreduced and memory can be saved.

Here, the transform parameters were described as being calculated whenwindow rotation processing was performed, but they may also be obtainedby performing calculation in advance using the rotation angle for thewindow, and storing the result in a table like the one in FIG. 40, whichmay be referred to when the window is rotated.

The present invention may also be realized using a standard personalcomputer.

Here, transform parameters were calculated using a similarity transform,but a window may also be transformed using a proportional calculationbased on the length of the edges, provided that such a system onlyrotates the window around either the y-axis or the x-axis.

Tenth Embodiment

The following is an explanation of a window display device in the tenthembodiment of the present invention.

The window display device in the present embodiment is related tocharacter display processing performed when a window is displayed inperspective, characterized by display processing performed so that eachcharacter always appears to be facing the front.

Structure

FIG. 43 is a block diagram showing a structure for a window displaydevice in the tenth embodiment of the invention. This embodiment hasmany similarities with the ninth embodiment, so only differing partswill be explained here. The basic differences are that the fonttransform unit 150 is replaced by a font scaling unit 1450, and thetransform parameter calculating unit 140 is replaced by a font scalecalculating unit 1440.

Once rotation processing has been completed, a display control unit 1470directs the perspective transform unit 130 to perform a perspectivetransform, and the font scaling unit 1450 to produce a character image,so that the two processes are performed in parallel.

The font scaling unit 1450 sends a z coordinate value from the charactercoordinates read from the character information unit 102 by a fontreading unit 1460 to the font scale calculating unit 1440, and has thefont scale calculating unit 1440 calculate a font scale from this value.The font scaling unit 1450 then performs scale processing on the fontbased on the font scale, producing a character image to be displayed ina 2D window. This means that the processing performed by the fonttransform unit 1450 is identical to that performed by the font transformunit 150 in the ninth embodiment, except for the font shape changingprocess.

Examples of character images produced by the font scaling unit 1450 areshown in FIGS. 44 and 45. FIG. 44 shows a 2D window 1500 produced byperforming a perspective transform on a 3D window on which no rotationprocessing has been implemented. FIG. 45 shows a 2D window 1600 producedwhen a perspective transform is performed on the same window as in FIG.44 after it has been rotated around the y-axis (edge AB) in a virtualspace.

Character strings 1501 and 1601 are displayed in the windows. Enlargedcharacter views 1502 and 1503 in FIG. 44 and 1602 and 1603 in FIG. 45each show one of the characters in the character string after it hasbeen enlarged, so as to clearly demonstrate the charactertransformation.

The enlarged character views 1602 and 1603 are simply reduced versionsof enlarged character views 1502 and 1503, which have undergone nochange in shape. As a result, if these characters are viewed inisolation no impression of depth is received. However, the size of thecharacters in the window becomes smaller the further they are from therotation axis, so that a impression of depth can be received by lookingat a character string as a whole.

The font scale calculating unit 1440 calculates font scale according toan input received from the font scaling unit 1450. This calculation isbased on a-z coordinate value from character coordinates for a transformobject character received together with the input. This processing isidentical to the processing performed by the font transform unit 150 inthe ninth embodiment, except that the processing for calculatingshape-changing parameters is omitted. The font scale unit 1440 informsthe font scaling unit 1450 of the calculated font scale.

Operation

FIG. 46 is a flowchart showing rotation processing for a windowdisplaying characters in the window display device in the tenthembodiment of in the present invention. The following is an explanationof this processing with reference to the flowchart. This processingdiffers from the ninth embodiment in several respects. Firstly, the fonttransform processing only involves changes in size produced by the fontscale and does not include shape-changing processing. In addition, theperspective transform processing and the font transform processing areperformed in parallel (in the ninth embodiment the font transformprocessing is performed after the perspective transform processing hasbeen completed) Processing performed up until rotation processing isperformed (steps S1301 and S1302 is the same as in the ninth embodiment.

Once rotation processing has been completed, the display control unit1470 directs the perspective transform unit 130 to execute theperspective transform processing, and instructs the font scaling unit1450 to execute processing to produce a character image for display useby transforming a font character. These two processes are performed inparallel (step S1701).

The content of the processing performed by the perspective transformunit 130 is the same as in the ninth embodiment (S1702).

Meanwhile the font scaling unit 1450 has the font reading unit 1460 readcharacter position coordinates (step S1703), sends the z coordinate fromthe coordinates to the font scale unit 1440 and has the font scalecalculating unit 1440 calculate the font scale (step S1704). Subsequentprocessing, in other words the processing up until a character image fordisplay use is produced by changing font size according to the fontscale, and sent to the display control unit 1470 (steps S1705 to S1707),is the same as steps S1306 to S1308 in the ninth embodiment, apart fromthe omission of the font shape-changing processing.

Processing for combining character images with a window on whichperspective transform has been performed (step S1309) is also the sameas in the ninth embodiment.

In the window display device of the present embodiment, a font characteris transformed by a font scale calculated based on information stored ina window information managing means, as explained above. As a result,there is no need to wait for the result of the perspective transformbefore transforming the font character, as was the case in the ninthembodiment. Instead, font transform processing is performed in parallelwith perspective transform processing, and font shape-changingprocessing is not performed, resulting in high processing efficiency.

Eleventh Embodiment

The following is an explanation of a window display device in theeleventh embodiment of the present embodiment.

The window display device in the present embodiment relates to displayprocessing for characters occurring when a window is displayed inperspective, and is characterized by displaying individual charactersfacing the front, while altering the aspect ratio for the outline ofeach character using a perspective transform.

Structure

FIG. 47 is a block diagram of the eleventh embodiment of the presentinvention. Explanation of those parts identical to the tenth embodimentwill be omitted.

Upon receiving an input from the display control unit 1470, a fonttransform unit 1850 sends a z coordinate from character positioncoordinates, obtained by instructing a font reading unit 1460, to a fontscale calculating unit 1440, and has the font scale calculation unit1440 calculate a font scale from this value (this process is identicalto the tenth embodiment). Meanwhile, the font transform unit 1850instructs an aspect ratio calculating unit 1810 to find an aspect ratio.

The font transform unit 1850 instructs the font reading unit 1460 toread a font character and changes the height to width ratio and size ofthe font character based on the aspect ratio and the font scale,producing a character image to be displayed in a 2D window.

Examples of the processing performed by the font transform unit 1850 areshown by FIGS. 48 and 49.

FIG. 48 shows a 2D window 1900 produced by performing a perspectivetransform on a 3D window on which no rotation processing has beenperformed. FIG. 49 shows a 2D window 2000 produced by performing aperspective transform on the same window as the one in FIG. 48 after ithas been rotated around the y-axis in virtual space.

Character strings 1901 and 2001 are displayed in the windows. Enlargedcharacter views 1902 and 1903 in FIG. 48 and 2002 and 2003 in FIG. 49display a character from the character string after it has beenenlarged, in order to demonstrate the character transform clearly.

Since rotation is performed around the y-axis, the aspect ratio has ascale factor of 1 in the y direction and a scale factor of a in the xdirection, where 0<α<1. If the output font scale based on the zcoordinate value of the character display position is y, and the size ofthe enlarged character views 1902 and 1903 is P, then an x directionlength C and a y direction length D for enlarged character views 2002and 2003 are expressed by the following formulas:

C=γ×α×P

 D=γ×1×P

where C is shorter than D.

The aspect ratio calculating unit 1810 reads a window angle from the 3Dinformation management unit 100, calculates an aspect ratio from thisangle, and informs the font transform unit 1850 of the aspect ratio. Theaspect ratio is the length to width ratio. An x direction scale and ydirection scale for a font character are calculated from the windowangle by the aspect ratio calculating unit 1810 and then output. Thescales are calculated according to various rotation angles, as shown inFIGS. 50A and 50B.

Operation

FIG. 51 is a flowchart showing rotation processing for a windowdisplaying characters in the window displaying device of the eleventhembodiment of this invention. The following is an explanation of thisprocessing with reference to the flowchart. Here, processing identicalto that in the tenth embodiment has the same effects and so explanationis omitted.

First, the procedure in which the control display unit 1470 receives aninput for performing a window rotation from a user using the input unit190, and has the rotation processing unit 120, perform rotationprocessing on coordinates (steps S1301, and S1302 is the same as in thetenth embodiment.

Upon receiving notice that the rotation processing has been completedfrom the rotation processing unit 120, the display control unit 1470outputs a perspective transform indication to the perspective transformunit 130, as well as instructing the font transform unit 1850 to changethe size of a font character and transmit it (step S1701).

The processing performed by the perspective transform unit 130 is thesame as in the tenth embodiment (step S1702).

Meanwhile, the font transform unit 1850 instructs the aspect ratiocalculating unit 1810 to find an aspect ratio. The aspect ratiocalculating unit 1810 reads a window angle from the 3D managementinformation unit 100, calculates the aspect ratio from this angle, andthen informs the font transform unit 1850 of this aspect ratio (stepS2201).

The font transform unit 1850 instructs the font reading unit 1460 toread coordinates for a character position in the window (step S1703).Then the font transform unit 1850 sends the z coordinate from thereceived character position coordinates to the font scale calculatingunit 1440, and has the font scale calculating unit 1440 calculate a fontscale from that value (step S1704). Upon receiving the font scale, thefont transform unit 1850 directs the font reading unit 1460 to read afont character corresponding to the character being processed from thefont database 110, and a font size from the character information unit102 (step S1705).

Then, the font transform unit 1850 performs a font transform processingbased on the obtained font size, font scale and aspect ratio, and sendsthe resulting character image to the display control unit 1470 togetherwith character position coordinates (step S2202).

The font transform unit 1850 repeats the above processing (from stepsS1703 to S2202) until no unprocessed characters remain. When all of thecharacters have been processed, the font transform unit 1850 informs thedisplay control unit 570 of this (step S1707).

The subsequent combining processing (step S1309) is the same as theprocessing in FIG. 10.

In the window display device of the present embodiment, font charactersare transformed according to an aspect ratio based on the rotation angleof the window, in addition to a font scale, as explained above. As aresult, when a character string is displayed on the display unit 180, amore natural impression of depth can be achieved than was the case inthe tenth embodiment when only the font size was changed.

The font scale calculating unit and the aspect ratio calculating unitmay also store a font scale and an aspect ratio correspondingrespectively to a z coordinate and a window rotation angle in memory.This data may be stored in memory as a table that has already beenquantized and that can be referred to when processing is performed.

Twelfth Embodiment

The following is an explanation of a window display device in a twelfthembodiment of the present invention.

The window display device in the present invention is related to displayprocessing for characters when a window is displayed in perspective, andcharacterized by having character font of various shapes on which aperspective transform has already been performed.

Structure

FIG. 52 is a block diagram of the twelfth embodiment of the presentinvention.

Transformed font characters for each font character are stored in a fontdatabase 2310. Suppose that each font character has a perspectivetransform performed on it after being rotated around the x or y-axis ina virtual space, in a similarity transform similar to that in the ninthembodiment.

FIGS. 53A to 53F are examples of such transformed font charactersobtained when a font character is rotated around the y-axis. FIGS. 53Aand 53D show examples when a rotation angle is 0°, FIGS. 53B and 53Ewhen a rotation angle is 30°, and FIGS. 53C and 53F when a rotationangle is 60°.

A font reading unit 2360 reads a transformed font character from thefont database 2310. The transformed font character corresponds to therotation angle nearest to a rotation angle that is an output value of arotation angle reading unit 2320. If a rotation angle output by therotation angle reading unit 2320 is 20°, for example, the font readingunit 2360 reads the transformed font character shown in FIG. 53B,corresponding to a rotation angle of 30°, and sends the read transformedfont character to a font transform unit 2350.

The rotation angle reading unit 2320 reads rotation angle informationfor a 3D window from window information unit 101, and informs the fonttransform unit 2350 of the value obtained.

The font transform unit 2350 directs the rotation angle reading unit2320 to find the size of the window rotation angle formed in the virtual3D space. Next, the font transform unit 2350 informs the font readingunit 2360 of the angle and directs the font reading unit 2360 to read acorresponding transformed font character from the font database 2310 andthe font size and character coordinates for this font character from thecharacter information unit 102. Having received this data, the fonttransform unit 2350 sends the z coordinate from the charactercoordinates to the font scale calculating unit 1440 and causes it tofind the font scale from this value.

The font transform unit 2350 implements scale processing, based on theobtained font scale and font size, on the transformed font obtained fromthe font reading unit 2360. The font transform unit 2350 then sends theresulting character image and corresponding character positioncoordinates to the display control unit 1470.

Operation

FIG. 54 is a flowchart showing the rotation processing for a windowdisplaying characters in the window display device in the twelfthembodiment of the present invention.

The following is an explanation of the rotation processing withreference to the flowchart. Processes that are the same as processes inthe tenth embodiment will be indicated as such and will not bedescribed.

Here, the procedures in which the display control unit 1470 receives aninput for a window rotation from a user via the input unit 190 and thefont transform unit 2350 initializes processing of a font character(steps S1301 and S1302 and the perspective transform processing (stepS1702) are identical to the tenth embodiment.

The font transform unit 2350 instructs the rotation angle reading unit2320 to calculate the window rotation angle (step S2501). Next, the fonttransform unit 2350 informs the font reading unit 2360 of the angle andinstructs the font reading unit 2360 to read a corresponding font sizeand character coordinates from the character information unit 102 (stepS1703). Having received this data, the font transform unit 2350 sendsthe z coordinate value from the character coordinates to the font scalecalculating unit 1440, and has the font scale calculating unit 1440calculate a font scale from this value (step S1704). Next, the fonttransform unit 2350 instructs the font reading unit 2360 to read a fontcharacter that corresponds to the angle read by the rotation anglereading unit 2320, and a character code and font type stored in thecharacter information unit 102, from the font database 2310 (stepS2502).

The font transform unit 2350 performs transform processing, based onfont scale and font size, on the font obtained from the font readingunit 2360, and sends the resulting character image and correspondingcharacter coordinates to the display control unit 1470 (step S2503).

The font transform unit 2350 repeats the above processing for eachcharacter in the window in turn, until no unprocessed characters remain.Once all of the characters have been processed, the font transform unit2350 informs the display control unit 1470 that processing has beencompleted (step S1707).

The subsequent combining processing (step S1309) is the same as in thetenth embodiment.

In the window display device of the present embodiment, font characterson which a transform has already been performed according to the windowrotation angle are stored in the font database 2310, as described above.Character images can be displayed in virtual 3D simply by changing thesize of the font characters in accordance with the font scale. Theshape-changing processing of the ninth embodiment is unnecessary, and sothe amount of processing can be reduced accordingly.

In the present embodiment, character sets that have been rotated atrepresentative angles are stored in advance and characters in the setthat best represents the window rotation angle are used. If the actualrotation angle does not equal any of the representative angles, atransformed character set corresponding to the actual angle may beformed by combining the two closest types of transformed font. If therotation angle is 20°, for example, the transformed font may be formedby combining FIG. 53A (or D) with FIG. 53B (or E).

Thirteenth Embodiment

The following is an explanation of a window display device in thethirteenth embodiment of the present invention.

The window display device in the present embodiment relates to displayprocessing for characters when a window is displayed in perspective, andis characterized by displaying a special fixed symbol for characterssmaller than a certain size.

Structure

FIG. 55 is a block diagram of the window display device in thethirteenth embodiment of the present invention. Explanation of thoseparts identical to the ninth embodiment is omitted.

A display judgement unit 2610 judges whether character images should bedisplayed, based on information from the 3D information management unit100. The aim of performing such judgements is explained below.

Thus far in this description, when a character image is displayed in avirtual 3D window, the size of the displayed character image isdetermined by the position of the character in virtual space (Zcoordinate) and font size, as explained in the ninth to twelfthembodiments. As a result, if characters are positioned in the back ofthe space (have a large z coordinate value) and have a small font size,displayed character images become smaller and impossible to read.Processing for producing and displaying such unreadable character imagesis unnecessary. Here, the size of character images is calculated inadvance based on character coordinates and font size and, if it isthought a character image will be unreadable, the font transform unit2650 is directed not to display it. In this case, a symbol or similar isoutput instead of the character image.

The display judgement unit 2610 calculates the length of the verticaledge of the character image from the font size and transform parameters(scale). following transformation, and performs judgement by comparingthe result with a threshold value.

The font transform unit 2650 instructs the font reading unit 2660 toread a character code, font size, font type and character coordinatesfrom the character information unit 102. Next the font transform unit2650 informs the display judgement unit 2610 of the font size and zcoordinate value from the character coordinates in this readinformation, along with the transform parameters, and has the displayjudgement unit 2610 judge whether the character should be displayed as acharacter image or replaced with a symbol. If the result of thejudgement is that the character should be replaced with a symbol, thefont transform unit 2650 sends a specified symbol to be displayedinstead of the character image to the display control unit 170 alongwith the character position coordinates for the character that is to bereplaced.

An example of the replacement of characters by symbols performed by thefont transform unit 2650 is explained with reference to FIGS. 56 and 57.FIG. 56 shows a perspective converted window 2700 prior to rotationprocessing, and FIG. 57 shows a perspective converted window 2800 onwhich rotation processing has been performed about the y-axis. Acharacter string 2701 is displayed in the window 2700 and a characterstring 2801 in the window 2800. The characters in the character string2801 decrease in size the further to the right they are (as the zcoordinate in the virtual space becomes larger). In each line anycharacters to the right of the tenth character are judged to beunreadable, and replaced by a black dot.

The display judgement unit 2610 finds the length of the vertical edgenearest to the rotation axis for each transformed character image fromthe font size and transform parameters (scale). The display judgementunit 2610 performs a judgement by comparing this length with a specifiedthreshold value stored in memory.

Operation

FIG. 58 is a flowchart showing the operation of rotation processing fora window including text fonts in the window display device of thethirteenth embodiment of the invention. The following is an explanationof the processing with reference to the flowchart. Here, processing thatis the same as in the ninth embodiment has the same content and soexplanation is omitted.

The processing up until the initialization of font transform processingby the font transform unit 2650, that is the processing from thereception of a window rotation input to the calculation of transformparameters (steps S1301 to S1304), is identical to the ninth embodiment.

The font transform unit 2650 directs the font reading unit 2660 to reada character code, font size, font type and character coordinates fromthe character information unit 102 (step S1305). Next, the fonttransform unit 2650 informs the display judgement unit 2610 of the fontsize, and the z coordinate from this information, along with thetransform parameters, and has the display, judgement unit 2610 judgewhether the character is to be replaced by a symbol (step S2901).

If the judgement result is that the character is to be replaced by asymbol, the font transform unit 2650 sends a specified symbol to thedisplay control unit 170 along with character position coordinates (stepS2902).

If the judgement result is that the character be displayed using a fontcharacter, subsequent processing is the same as in the ninth embodiment(steps S1306 and S1307).

The font transform unit 2650 repeats the above processing until nounprocessed characters remain, and once all the characters have beenprocessed, informs the display control unit 170 that processing has beencompleted (step S1308).

Subsequent processing (step S1309) is the same as in the ninthembodiment.

In the window display device of the present embodiment, characters thatare thought to be unreadable or difficult to read if transformed arereplaced by a black dot when displayed, enabling the font reading andfont transform processing (steps S1306 and S1307) to be omitted, so thatboth the amount of processing and processing time are reduced.

In the present embodiment, the length of the vertical edge nearest tothe rotation axis for the transformed character image was used as thejudgement criteria, but the width or the area of the character image mayalso be used as the criteria.

The present embodiment was explained in conjunction with the blockdiagram of the ninth embodiment, but the present embodiment may also beused in conjunction with the tenth, eleventh or twelfth embodiments. Thedisplay judgement unit 2610 forms judgements on the basis of font sizeand font scale when used in conjunction with the tenth and twelfthembodiments and on the basis of font size, font scale and aspect ratiowhen used in conjunction with the eleventh embodiment.

Fourteenth Embodiment

The following is an explanation of a window display device in thefourteenth embodiment of the present invention.

The window display device in the present embodiment is related todisplay processing for characters in a window displayed in perspective,and characterized by displaying characters smaller than a certain sizeusing specific fixed font.

Structure

FIG. 59 is a block diagram of a window display device in the fourteenthembodiment of the present invention. Parts with the same numericalreferences as in FIG. 30 have the same function and so explanation ofsuch parts is omitted.

A display judgement unit 3010 judges whether character images need to bedisplayed using a specified font type. The object of this process isexplained below.

When characters are. displayed in a virtual 3D window, the size ofcharacter images displayed is determined according to the position ofcharacters in virtual space (Z coordinate value) and font size, as wasalso explained in the thirteenth embodiment. As a result, if charactersare positioned in the back of the space, (have a large Z coordinatevalue) and have a small font size, the displayed character images becomesmaller. If the user cannot tell which font is being used, there is nopoint in using different font types. Here, if it is thought that thefont type will be unidentifiable if transformed, due to the characterdisplay position and size, and z coordinate value used, only charactersfrom a specified font type are used.

Using the above processing method improves processing efficiency for thefollowing reasons.

First, when font characters are read from the font database 110, aspecified font type group is stored in a cache memory. This means that,so long as the same type of font is being used, all reading takes placefrom the cache memory and, since the cache hit rate is high, the readspeed is improved. On the other hand, if the type of font being readchanges suddenly, the hit rate falls, and transmission from the fontdatabase unit 110 to the cache increases, lowering the read speed. As aresult, the processing method in the present embodiment uses thespecified font type for characters whose display font will not bediscernable, improving the cache hit rate and increasing the readprocessing speed.

As in the thirteenth embodiment, the display judgement unit 3010calculates the length of the vertical edge of a transformed characterimage from the font size and transform parameters, and makes a judgementby comparing the result with a threshold value.

Upon receiving an input from a font transform unit 3050, a font readingunit 3060, reads information concerning a character stored in thecharacter information unit 102, and sends this information to the fonttransform unit 3050.

Then, in response to an input from the font transform unit 3050, thefont reading unit 3060 reads either a font character corresponding tothe font type information in the character information unit 102, or tothe specified font type and sends this data to the font transform unit3050.

The font transform unit 3050 instructs the font reading unit 3060 toread a character code, font size, font type, and character coordinatesfrom the character information unit 102. Next, the font transform unit3050 informs the display judgement unit 2710 of the font size and zcoordinate value from the character coordinates in this information,along with transform parameters. If the character has been transformed,the font transform unit 3050 has the font judgement unit 2710 judgewhether the character is of sufficient size for the font type to beidentified.

If the result of the judgement is that identification will beimpossible, the font transform unit 3050 instructs the font reading unit3060 to ignore the font type information obtained from the characterinformation unit 102, and read the font character corresponding to thecharacter from the specified font type group.

An example of the font type replacement performed by the font transformunit 3050 is shown in FIGS. 60 and 61.

FIG. 60 shows a perspective converted window 3100 prior to rotationprocessing. FIG. 61 shows a perspective converted window 3200 which hasbeen rotated around the y-axis. A character string 3101 is displayed inthe window 3100 and a character string 3201 in the window 3200. Thecharacter string 3201 has four lines, the characters in the first andsecond lines having a different font type to those in the third andfourth lines. The character string 3201 in the window 3200 becomessmaller as it moves further to the right (as the z coordinate value inthe virtual space becomes larger). A part 3202 of the character string3201, for which the font type is judged to be unidentifiable, isdisplayed with characters from the specified font type, withoutdistinguishing between font types.

Operation

FIG. 62 is a flowchart showing the operation of rotation processing fora window displaying characters in a window display device in thefourteenth embodiment of the invention. The following is an explanationof the processing with reference to the flowchart. Here, processing thatis the same as in the ninth embodiment has the same effects and soexplanation is omitted.

The processing up until the initialization of font transform processingby the font transform unit 3050, that is the processing from thereception of a window rotation input to the calculation of transformparameters (steps S1301 to S1304), is identical to the ninth embodiment.

The font transform unit 3050 instructs the font reading unit 3060 toread a character code, font size, font type and character coordinatesfrom the character information unit 102 (step S1305). Next, the fonttransform unit 3050 informs the display judgement unit 3010 of the fontsize, and the z coordinate value from the character coordinates fromthis information, along with the transform parameters, and if thecharacter has been transformed, has the display judgement unit 3010judge whether the character is large enough for the font type to beidentified (step S3301).

If identification is judged to be impossible, the font transform unit3050 instructs the font reading unit 3060 to ignore the font typeinformation obtained from the character information unit 102, and readthe font character from the specified font type group (step S3302).

If identification is judged to be possible, the font transform unit 3050directs the font reading unit 3060 to read the font character from afont type group in the character information unit 102, as was the casein other embodiments (step S1306). The subsequent processing is the sameas in the thirteenth embodiment (step S1307).

The font transform unit 3050 repeats the above processing until nounprocessed characters remain, and when all characters have beenprocessed, informs the display control unit 170 of the completion ofprocessing (step S1308).

Subsequent combining processing is identical to that in the thirteenthembodiment (step S1309).

In the window display device of the present embodiment, the individualfont types are ignored for characters thought to have an unidentifiablefont type if transformed. Such characters are displayed using a constantspecified font type. As a result, if a cache memory is used, the cachehit rate for font read processing improves and processing efficiency israised.

Here, if the font type is judged to be unidentifiable, the fontcharacter was to be read from the specified font type group, but it mayalso be read from the font type group stored in the cache memory at thatpoint in time, thus further improving the cache hit rate.

The replacement of font by the font transform unit 3050 was mandatory,but, alternatively, the font type may be left unchanged, and a fontcharacter may be displayed after certain elements used to construct ithave been omitted.

The present embodiment, was explained in conjunction with the blockdiagram of the ninth embodiment, but the present embodiment may also beused in conjunction with the tenth, eleventh or twelfth embodiments. Thedisplay judgement unit 3010 forms judgements on the basis of font sizeand font scale when used in conjunction with the tenth and twelfthembodiments, and on the basis of font size, font scale, and an aspectratio when used in conjunction with the eleventh embodiment.

Fifteenth Embodiment

The following is an explanation of a window display device in thefifteenth embodiment of the present invention.

The window display device in the present embodiment is related todisplay processing for characters when a window is displayed inperspective, and characterized by the ability to change the scale of acoordinate axis so that characters are displayed with a fixed width evenif they are positioned deep within the 3D space.

The processing order in the present embodiment is the same as in otherembodiments, except for the way in which a 2D window coordinate systemis changed prior to rotation processing, so explanation of the operationusing a flowchart is omitted.

FIG. 63 is a block diagram of the window display device in the fifteenthembodiment of the present invention. Parts with the same referencenumerals as those in FIG. 30 have an identical function, so explanationof these parts is omitted. In FIG. 63, a coordinate system convertingunit 3410 changes the ratio (scale of the coordinate axis) of a screencoordinate system displayed in a window, based on a window position androtation angle read from the 3D information management unit 100. Thechange is timed to be performed immediately before the perspectivetransform processing. The coordinate system data is stored in themiscellaneous information unit 103.

The coordinate system converting unit 3410 converts the coordinatesystem in the window in the following way. First, the coordinate systemconverting unit 3410 obtains window rotation information from the 3Dinformation management unit 100. Then, if the new screen coordinatesystem is (X, Y), the coordinate system converting unit 3410 convertsthe coordinate system so that$\left( {X,Y} \right) = {\frac{1}{\alpha}\left( {x,y} \right)}$

where the value of a is set according to the rotation angle. As aresult, the display area in the window space looks like that shown inFIG. 65C, and after rotation, like that in FIG. 65D.

This situation is explained with reference to FIGS. 64 and 65.

FIG. 64A is a perspective converted view of a window with a coordinatesystem using prior art techniques, prior to rotation processing. FIG.64B is a perspective converted view of a window with a prior artcoordinate system, on which rotation processing has been performedaround the x-axis. FIG. 64C is a perspective converted view of a windowwith the converted coordinate system, prior to rotation processing. FIG.64D is a perspective converted view of a window with the convertedcoordinate system, on which rotation processing has been performedaround the x-axis.

If a perspective transform is performed on the window in FIG. 64A afterit has been rotated around the x-axis, the upper part of the windowshown in FIG. 64B will be displayed much larger than the lower part.

FIG. 65 shows window coordinate systems. FIG. 65A is a window coordinatesystem prior to rotation, and FIG. 65B is a coordinate system when thewindow is rotated about the x-axis. When rotation is performed about thex-axis, the upper part of the window appears larger and the lower partsmaller, as shown in FIG. 65B. Here, when an x-axis rotation isperformed, the window coordinate system is already set as the one shownin FIG. 65C. This transform is performed according to a ratio calculatedbased on the rotation angle. By converting the coordinate system, whenan x-axis rotation is performed the upper and lower parts of the windowhave almost the same display area, as shown in FIG. 65D. This isillustrated by an actual window in FIGS. 64C and 64D. In FIG. 64D, theproportion of the window in which the menu is displayed is smaller andthat in which the main text is displayed larger than in FIG. 64B, and socharacters in the lower part of the window are much easier to read.

In the window display device of the present embodiment, changing theratio of the coordinate system enables characters far from the rotationaxis, which would have become tightly packed and difficult to read in anormal coordinate system, to be displayed so that they are readilydecipherable.

The value of a need not be changed based only on the rotation angle. Ifthe title bar and menu bar part of the window, for example, are judgednot to require such a large amount of display space, the coordinatesystem may be changed only for this part of the window, enabling thedisplay area of the window to be used effectively.

In the present embodiment, explanation took place in conjunction withthe block diagram of the ninth embodiment, but the present embodimentmay also be used in conjunction with the tenth, eleventh or twelfthembodiments.

In the ninth to fifteenth embodiments, each character font is aconventional font (one pixel is expressed by one bit). However, thepresent invention may also be applied when characters are displayedusing a gray scale font (density for one pixel is expressed inmulti-level data expressed as one of three or more values). As shown inFIGS. 66A and B, curved parts of the characters appear smoother and morenatural when represented by gray scale fonts (FIG. 66B) rather thanconventional fonts (FIG. 66A). Two font types such as these may beapplied selectively in the thirteenth and fourteenth embodiments. In thethirteenth embodiment, for example, characters smaller than a certainsize are represented by a specified symbol (FIG. 66C), but characters ofa medium size may be represented using a gray scale font. This meansthat conventional font can be used for large characters without obviousjagged curves, a gray scale font for medium characters which would haveobvious jagged curves if expressed by a conventional font, and specifiedsymbols for characters that are too small to be readable. As a result,display processing of the characters can be completed in a short timeusing less memory, without any deterioration in the quality andattractiveness of character display.

In the above embodiments, windows are rectangular planes, but need notbe limited to this shape. Windows may themselves have depth or bepolygonal in shape. This is because the display screen of a window maybe projected so that it appears in perspective, regardless of its shape.The present invention may of course be applied in cases such as FIG.67A, in which the faces of a cuboid correspond to a plurality ofwindows, and FIG. 67B, in which a window is displayed on the cover of abook.

As explained above, the window display device of the present inventiondisplays windows so that they appear to be in perspective. As a result,the display area of a window placed in perspective is less than thatrequired to display a window facing the front in a conventionalwindowing system, enabling the display area of the screen to be usedmore effectively. Accordingly, the present invention can be applied to aportable information terminal, such as the one in FIG. 68A, which needsto display a large number of windows on a small screen, and to awindowing system for a computer system providing a high-performance GUI.

Industrial Applicability

The invention may be used as a man-machine interface in electronicdevices such as personal computers and information processing terminals.It is particularly suited for use as a windowing system interacting witha user using a plurality of windows in a portable information terminalwith a small screen.

What is claimed is:
 1. A window display device displaying a plurality ofwindows on a screen, the window display device comprising: a displaydata holding means for holding display data representing display contentof the plurality of windows; a position determining means fordetermining a position for each window in a virtual three-dimensional(3D) space, the positions being such that the windows are not parallelto the screen, wherein the position determining means determines 3Dcoordinates for each vertex of a window as the position for the window;an input means for acquiring an indication from a user, wherein theposition determining means determines positions according to indicationsacquired from the input means; an window analyzing means for detecting,as a key part for each window, a part of the display content fulfillingcertain conditions, detection being performed based on display data heldby the display data holding means, wherein the position determiningmeans determines the position of windows so that the key parts detectedby the window analyzing means are displayed near the front of thevirtual 3D space; a perspective projection means for transforming thedisplay data to project the windows onto the screen in perspective basedon the determined positions; and a display means for displaying thetransformed display data on the screen.
 2. The window display device ofclaim 1, wherein the window analyzing means determines whether a titleis included in the display data for each window and, if a title isincluded, detects a part of a window in which the title is displayed asthe key part.
 3. The window display device of claim 1, wherein thewindow analyzing means determines whether a text and an indication fordisplaying the text horizontally are included in the display data foreach window, and if the text and the indication are included, detects aleft side of a window when viewed from the front as the key part.
 4. Thewindow display device of claim 1, wherein the window analyzing meansdetermines whether a text, and an indication for displaying the textvertically are included in the display data for each window, and if thetext and the indication are included, detects a right side of a windowwhen viewed from the front as the key part.
 5. The window display deviceof claim 1, wherein the position determining means includes an automaticalignment unit for determining the position of a plurality of windows sothat the windows are placed in a predetermined arrangement, where atleast one window is not parallel with the screen.
 6. The window displaydevice of claim 5, wherein the position determining means furtherincludes: a means for determining a position according to an indicationacquired from the input means, so that a window displayed in thealignment position is newly displayed facing the front; and a means fordetermining a position according to an indication acquired from theinput means, so that a window displayed facing the front is returned tothe alignment position.
 7. The window display device of claim 1, furthercomprising a window information image position changing means forediting display data stored in the display data holding means so that awindow information image is displayed near the front of the virtual 3Dspace, the window information image being formed from a title bar and amenu bar, and editing being performed based on a position determined bythe position determining means.
 8. The window display device of claim 1,further comprising a window information image orientation changing meansfor controlling the position determining means so that, in a windowwhere a window information image is displayed near the front of thevirtual 3D space, the window information image faces the front, whilethe remaining part of the window is not parallel with the screen.
 9. Thewindow display device of claim 1, further comprising: an unreadable areadetermining means for determining whether an unreadable area displayingcharacters smaller than a specified size has been generated from displaydata transformed by the perspective projection means; a scroll buttonposition determining means for determining a position for a scrollbutton when an unreadable area has been determined to exist, theposition determined as if the unreadable area was not displayed in thewindow; and a scroll bar attachment means for editing the display dataso that a scroll bar image, including a scroll button arranged at thedetermined position, is attached to the window.
 10. The window displaydevice for claim 1, further comprising: a layer separating means forseparating the display data held in the display data holding means, thedisplay data corresponding to a window with a layered constructionhaving a base layer and at least one upper layer, and the display datafor each upper layer, but not the base layer, being separated; and awindow layer generating means for generating display data for a windowlayer corresponding to each separated layer, wherein the positiondetermining means determines a position for each window layer parallelwith and a specified distance from the base layer.
 11. The windowdisplay device of claim 1, further comprising: a keyword searching meansfor searching display data held in the display data holding means forparts that match a keyword acquired from the input means, and a windowlayer generating means for generating display data for a new windowlayer in which the keyword is arranged at positions corresponding to thematching parts, and storing the generated display data in the displaydata holding means, wherein the position determining means determines aposition for the window layer parallel with and a specified distancefrom a display object window.
 12. The window display device of claim 1,further comprising: an inter-related window specifying means forspecifying a group of inter-related windows from the plurality ofdisplayed windows; and a link information storage means for generatingand storing link information used to line up the group of inter-relatedwindows in a concertina arrangement with frames of neighboring windowstouching, wherein the position determining means determines positions ofthe group of inter-related windows based on the link information. 13.The window display device of claim 12, further comprising: a linkinformation changing means for changing the link information accordingto an indication acquired from the input means, so that the group ofinter-related windows is lined up with part of the group excluded,wherein the position determining means determines positions for thegroup of inter-related windows, apart from the excluded part of thegroup, based on the changed link information.
 14. The window displaydevice of claim 13, further comprising: an inter-window editing meansfor editing display data held in the display data holding meansaccording to an indication acquired from the input means, so thatediting of display content can be performed between two windows in thegroup of inter-related windows.
 15. A window displaying devicedisplaying a plurality of windows on a screen, the window display devicecomprising: a display data holding means for holding display datarepresenting display content of the plurality of windows; a positiondetermining means for determining a position for each window in avirtual three-dimensional (3D) space, by determining 3D coordinates foreach vertex of a window as the position for the window, the positionsbeing such that the windows are not parallel to the screen; an inputmeans for acquiring an indication from a user, wherein the positiondetermining means determines positions according to indications acquiredfrom the input means; a window information image orientation changingmeans for controlling the position determining means so that, in awindow where a window information image is displayed near the front ofthe virtual 3D space, the window information image faces the front,while the remaining part of the window is not parallel with the screen;a perspective projection means for transforming the display data toproject the windows onto the screen in perspective based on thedetermined positions; and a display means for displaying the transformeddisplay data on the screen.
 16. A window display device displaying aplurality of windows on a screen, the window display device comprising:a display data holding means for holding display data representingdisplay content of the plurality of windows; a position determiningmeans for determining a position for each window in a virtualthree-dimensional (3D) space, by determining 3D coordinates for eachvertex of a window as the position for the window, the positions beingsuch that the windows are not parallel to the screen; an input means foracquiring an indication from a user, wherein the position determiningmeans determines positions according to indications acquired from theinput means; a layer separating means for separating display data heldin the display data holding means, the display data corresponding to awindow with a layered construction having a base layer and at least oneupper layer, but not the base layer, being separated; a window layergenerating means for generating display data for a window layercorresponding to each separated layer, wherein the position determiningmeans determines a position for each window layer parallel with and aspecified distance from the base layer; a perspective projection meansfor transforming the display data to project the windows onto the screenin perspective based on the determining positions; and a display meansfor displaying the transformed display data on the screen.
 17. A windowdisplay device displaying a plurality of windows on a screen, the windowdisplay device comprising: a display data holding means for holdingdisplay data representing display content of the plurality of windows; aposition determining means for determining a position for each window ina virtual three-dimensional (3D) space, by determining 3D coordinatesfor each vertex of a window as the position for the window, thepositions being such that the windows are not parallel to the screen; aninput means for acquiring an indication from a user, wherein theposition determining means determines positions according to indicationsacquired from the input means a keyword searching means for searchingdisplay data held in the display data holding means for parts that matcha keyword acquired from the input means, and a window layer generatingmeans for generating display data for a new window layer in which thekeyword is arranged at positions corresponding to the matching parts,and storing the generated display data in the display data holdingmeans, wherein the position determining means determines a position forthe window layer parallel with and a specified distance from a displayobject window; a perspective projection means for transforming thedisplay data to project the windows onto the screen in perspective basedon the determined positions; and a display means for displaying thetransformed display data on the screen.
 18. A window display devicedisplaying a plurality of windows on a screen, the window display devicecomprising: a display data holding means for holding a display datarepresenting display content of the plurality of windows; a positiondetermining means for determining a position for each window in avirtual three-dimension (3D) space, by determining 3D coordinates foreach vertex of a window as the position for the window, the positionsbeing such that the windows are not parallel to the screen; an inputmeans for acquiring an indication from a user, wherein the positiondetermining means determines positions according to indications acquiredfrom the input means a perspective projection means for transforming thedisplay data to project the windows onto the screen in perspective basedon the determined positions; and a display means for displaying thetransformed display data on the screen, wherein the display data holdingmeans includes: a character data storage unit storing character dataformed of character information displayed in the windows; and a windowdata storage unit storing window data formed of information concerningnon-character information displayed in the windows, wherein the positionplacement determining means determines a position for each window andthe corresponding characters according to an indication acquired fromthe input means, and the perspective projection means includes: aperspective transform unit generating two-dimensional data showing thevertices and window image data by performing a perspective projectiontransform on the window data, the perspective projection transform beingperformed based on window positions determined by the positiondetermining means; a character data transform unit generating characterimage data by transforming the character data, based on characterpositions determined by the position determining unit andtwo-dimensional data showing vertices obtained by the transformperformed by the perspective transform unit; and a combining unitcombining image data generated by the perspective transform unit andimage data generated by the character data transform unit, the displaymeans displaying image data combined by the combining unit, wherein thecharacter data storage unit includes: a character information storageunit storing information relating to characters displayed in the window,the information relating to characters displayed in the window, theinformation including at least character position information showingpositions of the characters in virtual space, character identifyinginformation specifying which characters should be displayed, andcharacter size information; and a character image storage unit storingcharacter images, wherein the window data storage unit stores graphicsdata, comprising window display content excluding characters, and windowcoordinate information showing a position of the window in the virtual3D space, the position determining means changes the window data and thecharacter position information in response to an indication acquiredfrom the input means; the perspective transform unit performs aperspective projection transform on window data that has been changed bythe position determining means, the character data transform unitincludes: a character position information conversion unit convertingcharacter position information changed by the position determining meansinto screen position information; a character image selecting unitselecting a character image from the character images stored in thecharacter image storage unit, the character image corresponding to thecharacter identifying information; and a character image transform unittransforming the character image selected by the character imageselecting unit in accordance with the result of the position changingprocess performed by the position determining means, wherein thecombining unit combines a character image transformed by the characterimage transform unit with image data generated by the perspectivetransform unit, so that the character image is placed at a position inthe window shown by the screen position information produced by thecharacter position information conversion unit, the window having beentransformed onto the screen by the perspective transform unit, whereinthe character image storage unit stores character images on which asimilarity transform has been performed for each of a plurality ofrepresentative positions for a window in the virtual 3D space, thesimilarity transform being related to the shape of the window producedat each of the plurality of positions as a result of a perspectiveprojection transform; the character image selecting unit selects acharacter image from the character image storage unit, the characterimage coinciding with character identifying information stored in thecharacter information storage unit, and with window coordinateinformation stored in the window data storage unit; and the characterimage transform unit finds a scale based on character positioninformation stored in the character information storage unit, andchanges the size of the character image selected by the character imageselecting unit, the size of the character image being changed accordingto the scale.
 19. The window display device of claim 18, furthercomprising a coordinate system converting means for changing a scale ofa coordinate axis showing screen positions, the scale being changedaccording to window coordinate information stored in the window datastorage unit, wherein the perspective projection unit performs aperspective projection transform, converting the coordinate system to acoordinate system having a coordinate axis with the changed scale.
 20. Awindow display device displaying a plurality of windows on a screen, thewindow display device comprising: a display data holding means forholding a display data representing display content of the plurality ofwindows; a position determining means for determining a position foreach window in a virtual three-dimensional (3D) space, by determining 3Dcoordinates for each vertex of a window as the position for the window,the positions being such that the windows are not parallel to thescreen; an input means for acquiring an indication from a user, whereinthe position determining means determines positions according toindications acquired from the input means a perspective projection meansfor transforming the display data to project the windows onto the screenin perspective based on the determined positions; and a display meansfor displaying the transformed display data on the screen, wherein thedisplay data holding means includes: a character data storage unitstoring character data formed of character information displayed in thewindows; and a window data storage unit storing window data formed ofinformation concerning non-character information displayed in thewindows, wherein the position placement determining means determines aposition for each window and the corresponding characters according toan indication acquired from the input means, and the perspectiveprojection means includes: a perspective transform unit generatingtwo-dimensional data showing the vertices and window image data byperforming a perspective projection transform on the window data, theperspective projection transform being performed based on windowpositions determined by the position determining means; a character datatransform unit generating character image data by transforming thecharacter data, based on character positions determined by the positiondetermining unit and two-dimensional data showing vertices obtained bythe transform performed by the perspective transform unit; and acombining unit combining image data generated by the perspectivetransform unit and image data generated by the character data transformunit, the display means displaying image data combined by the combiningunit, wherein the character data storage unit includes: a characterinformation storage unit storing information relating to charactersdisplayed in the window, the information relating to charactersdisplayed in the window, the information including at least characterposition information showing positions of the characters in virtualspace, character identifying information specifying which charactersshould be displayed, and character size information; and a characterimage storage unit storing character images, wherein the window datastorage unit stores graphics data, comprising window display contentexcluding characters, and window coordinate information showing aposition of the window in the virtual 3D space, the position determiningmeans changes the window data and the character position information inresponse to an indication acquired from the input means; the perspectivetransform unit performs a perspective projection transform on windowdata that has been changed by the position determining means, thecharacter data transform unit includes: a character position informationconversion unit converting character position information changed by theposition determining means into screen position information; a characterimage selecting unit selecting a character image from the characterimages stored in the character image storage unit, the character imagecorresponding to the character identifying information; and a characterimage transform unit transforming the character image selected by thecharacter image selecting unit in accordance with the result of theposition changing process performed by the position determining means,wherein the combining unit combines a character image transformed by thecharacter image transform unit with image data generated by theperspective transform unit, so that the character image is placed at aposition in the window shown by the screen position information producedby the character position information conversion unit, the window havingbeen transformed onto the screen by the perspective transform unit,further comprising a character image readability judging means for (1)calculating a size for a transformed character image transformed by thecharacter image transform unit from (a) size information stored in thecharacter information storage unit, and (b) a scale calculated fromcharacter position information stored in the character informationstorage unit, (2) judging whether the transformed character image isreadable by comparing the calculated size with a threshold value, and(3) informing the character image selecting unit of the judgment result,wherein the character image storage unit further stores symbol images;the character image selecting unit selects a specified symbol image fromthe character image storage unit to be displayed instead of a characterimage if the character image readability judging means judges that thecharacter is unreadable; the character image transform unit does notperform transform processing on the symbol image, and the combining unitcombines the symbol image with image data generated by the perspectivetransform unit so that the symbol image is placed at a position in awindow shown by screen position information produced by the characterposition information conversion unit, the window having been transformedonto the screen by the perspective transform unit.
 21. The windowdisplay device of claim 20, wherein the character image transform unitincludes: a shape transform unit transforming the shape of the characterimage using a similarity transform related to the shape of the windowtransformed by the perspective transform unit; and a scaling unitfinding a scale based on character position information stored in thecharacter information storage unit, and changing the size of thecharacter image according to the scale.
 22. The window display device ofclaim 20, wherein the character image transform unit calculates a scalebased on character position information stored in the characterinformation storage unit, and changes the size of the character imageaccording to the scale.
 23. The window display information of claim 20,wherein the character image transform unit includes: an aspect ratioconversion unit finding an aspect ratio based on character positioninformation stored in the character information storage unit andchanging the length to width ratio of the character image according tothe aspect ratio; and a scaling unit finding a scale based on characterposition information stored in the character information storage unit,and changing the size of the character image according to the scale. 24.A window display device displaying a plurality of windows on a screen,the window display device comprising: a display data holding means forholding a display data representing display content of the plurality ofwindows; a position determining means for determining a position foreach window in a virtual three-dimensional (3D) space, by determining 3Dcoordinates for each vertex of a window as the position for the window,the positions being such that the windows are not parallel to thescreen; an input means for acquiring an indication from a user, whereinthe position determining means determines positions according toindications acquired from the input means a perspective projection meansfor transforming the display data to project the windows onto the screenin perspective based on the determined positions; and a display meansfor displaying the transformed display data on the screen, wherein thedisplay data holding means includes: a character data storage unitstoring character data formed of character information displayed in thewindows; and a window data storage unit storing window data formed ofinformation concerning non-character information displayed in thewindows, wherein the position placement determining means determines aposition for each window and the corresponding characters according toan indication acquired from the input means, and the perspectiveprojection means includes: a perspective transform unit generatingtwo-dimensional data showing the vertices and window image data byperforming a perspective projection transform on the window data, theperspective projection transform being performed based on windowpositions determined by the position determining means; a character datatransform unit generating character image data by transforming thecharacter data, based on character positions determined by the positiondetermining unit and two-dimensional data showing vertices obtained bythe transform performed by the perspective transform unit; and acombining unit combining image data generated by the perspectivetransform unit and image data generated by the character data transformunit, the display means displaying image data combined by the combiningunit, wherein the character data storage unit includes: a characterinformation storage unit storing information relating to charactersdisplayed in the window, the information relating to charactersdisplayed in the window, the information including at least characterposition information showing positions of the characters in virtualspace, character identifying information specifying which charactersshould be displayed, and character size information; and a characterimage storage unit storing character images, wherein the window datastorage unit stores graphics data, comprising window display contentexcluding characters, and window coordinate information showing aposition of the window in the virtual 3D space, the position determiningmeans changes the window data and the character position information inresponse to an indication acquired from the input means; the perspectivetransform unit performs a perspective projection transform on windowdata that has been changed by the position determining means, thecharacter data transform unit includes: a character position informationconversion unit converting character position information changed by theposition determining means into screen position information; a characterimage selecting unit selecting a character image from the characterimages stored in the character image storage unit, the character imagecorresponding to the character identifying information; and a characterimage transform unit transforming the character image selected by thecharacter image selecting unit in accordance with the result of theposition changing process performed by the position determining means,wherein the combining unit combines a character image transformed by thecharacter image transform unit with image data generated by theperspective transform unit, so that the character image is placed at aposition in the window shown by the screen position information producedby the character position information conversion unit, the window havingbeen transformed onto the screen by the perspective transform unit,wherein the character information storage unit further stores font typeinformation showing character font types; the character image storageunit stores character images for each font type; and the window displaydevice further includes a character image readability judging means for(1) calculating a size for a transformed character image transformed bythe character image transform unit from (a) size information stored inthe character information storage unit, and (b) a scale calculated fromcharacter position information stored in the character informationstorage unit, (2) judging if the font type of the transformed characterimage is readable by comparing the calculated size with a thresholdvalue, and (3) informing the character image selecting unit of thejudgment result, wherein, when the font type of a character is judgedunreadable by the character image readability judging means, thecharacter image selecting unit selects a character image with aspecified font type, ignoring font type information stored in thecharacter information storage unit.